KERAS_技巧

微调Fine_tune:

导入keras标准模型：

model_pretrained= ResNet50(weights='imagenet', include_top=False, input_shape=(768, 768, 3))

或者导入自己训练的模型：

model_pretrained=load_model(“**.hdf5”)

以上这个模型默认有初始化权值

model_new= ResNet50(weights=None, include_top=False, input_shape=(768, 768, 1)) 

这个模型没有初始化权值，需要从上一个模型中加载需要层的权重

除最后一层外，权重全部导入

for i in range(len(model_ pretrained.layers)-1):

    model_new.layers[i].set_weights(model_pretrained.layers[i].get_weights())

或者(不需要导入weights的层的名字要改掉)：

model_pretrained.load_weights("**.hdf5",by_name=True)

 

固定特定层权重freeze weights:

base_model = InceptionV3(weights='imagenet', include_top=False) 

for layer in base_model.layers:  

    layer.trainable = False

或者比如指定前3层不训练

for layer in base_model.layers[:3]:  

   layer.trainable = False

 

图片预处理ImageDataGenerator

fromkeras.preprocessing.image import ImageDataGenerator, array_to_img,img_to_array, load_img

datagen = ImageDataGenerator(
rotation_range=10,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
vertical_flip=True,
fill_mode='nearest')
"""
rotation_range：整数，数据提升时图片随机转动的角度
width_shift_range：浮点数，图片宽度的某个比例，数据提升时图片水平偏移的幅度
height_shift_range：浮点数，图片高度的某个比例，数据提升时图片竖直偏移的幅度
shear_range：浮点数，剪切强度（逆时针方向的剪切变换角度）
zoom_range：浮点数或形如[lower,upper]的列表，随机缩放的幅度，若为浮点数，则相当于[lower,upper] = [1 - zoom_range, 1+zoom_range]
horizontal_flip：布尔值，进行随机水平翻转
vertical_flip：布尔值，进行随机竖直翻转
fill_mode：；‘constant’，‘nearest’，‘reflect’或‘wrap’之一，当进行变换时超出边界的点将根据本参数给定的方法进行处理
"""

img =load_img('./data/cat.jpg')  # this is aPIL image
x = img_to_array(img)  # this is a Numpyarray with shape (3, 150, 150)
x = x.reshape((1,) + x.shape)  # this isa Numpy array with shape (1, 3, 150, 150)

i = 0
for batch in datagen.flow(x, batch_size=1,save_to_dir='./data/',save_prefix='cat', save_format='jpg'):
i += 1
    if i> 20:
        break

 

模型训练ModelCheckpoint,LearningRateScheduler,ReduceLROnPlateau, EarlyStopping, TensorBoard,CSVLogger

from keras.callbacksimportModelCheckpoint,LearningRateScheduler,ReduceLROnPlateau,EarlyStopping, TensorBoard

def schedule_decay(epoch):

initial_lrate = 0.01

   drop = 0.5

epochs_drop = 10.0

lrate = initial_lrate *math.pow(drop,math.floor((1+epoch)/epochs_drop))

   return lrate

checkpointer=keras.callbacks.ModelCheckpoint(filepath='name_epoch{epoch:02d}_valacc{val_acc:.2f}_valloss{val_loss:.2f}.hdf5',monitor='val_acc',verbose=1, save_best_only=True, save_weights_only=False,mode='auto', period=50)

decaylr=keras.callbacks.LearningRateScheduler(schedule_decay)

reducelr=keras.callbacks.ReduceLROnPlateau(monitor='val_loss',factor=0.1, patience=4, verbose=0, mode='auto', epsilon=0.0001, cooldown=0,min_lr=0)

earlystop=keras.callbacks.EarlyStopping(monitor='val_loss', patience=20, verbose=1, mode='auto')

tensorboard=keras.callbacks.TensorBoard(log_dir='./logs', histogram_freq=0, write_graph=True, write_images=False, embeddings_freq=0,embeddings_layer_names=None, embeddings_metadata=None)

csvlog = keras.callbacks.CSVLogger('./log.csv')

model.fit_generator(datagen.flow(X_train,Ytrain_cate,batch_size=batch_size),

samples_per_epoch=train_batch_size, \

nb_epoch=epoch, \

validation_data=(X_val, Y_val) \

nb_val_samples=self.val_batch_size, \

verbose=1, nb_worker=1, \

callbacks=[checkpointer,decaylr,reducelr,earlystop,tensorboard,csvlog])

模型可视化graphviz，pydot-ng

Python3 版本

mac:

brew install graphviz

pip install pydot-ng

 

ubuntu:

pip3 install graphviz

pip3 install pydot-ng

 

windows:

1，首先官网下载，http://www.graphviz.org/Download_windows.php

本人下载的是zip压缩包，解压后，将bin添加进环境变量，比如，本人将E:\graphviz-2.38\bin添加进环境变量

2，pip  install graphviz

3，pip  install pydot-ng

4，进入E:\Anaconda3\Lib\site-packages\pydot_ng，修改__init__.py中#Method 3 (Windows only)，将自己的graphviz 中bin路径添加进来，path = r"E:\graphviz-2.38\bin"

5，测试：

#encoding:utf-8

from keras.models import *

from keras.layers import *

from keras.utils.vis_utils importplot_model

model = Sequential()

model.add(Conv2D(32, (3, 3),activation='relu', input_shape=(100, 100, 3)))

model.add(MaxPooling2D(pool_size=(2,2)))

model.add(Flatten())

model.add(Dense(256, activation='relu'))

model.add(Dropout(0.5))

model.add(Dense(10,activation='softmax'))

plot_model(model,to_file='model.png',show_shapes=True)

或者使用HDFView，也可以进行模型的可视化，并且不需要编程，只要有模型就可以https://www.hdfgroup.org/downloads/hdfview/

设置调用cpu,gpu

os.environ["CUDA_VISIBLE_DEVICES"]= "-1"#调用cpu

os.environ["CUDA_VISIBLE_DEVICES"]= "0"#调用gpu0

os.environ["CUDA_VISIBLE_DEVICES"]= "1"#调用gpu1

 

释放tensorflow会话session

from keras import backend as K

K.clear_session()

Keras 模型压缩(SVD方法)

https://github.com/DwangoMediaVillage/keras_compressor

service服务注意

keras:

global thread:

   self.model = load_model(model_path)

   self.model._make_predict_function()

   self.graph = tf.get_default_graph()

another thread:

   with self.graph.as_default():

       labels = self.model.predict(data)

tensorflow：

global thread:

   self.graph = tf.Graph()

   with self.graph.as_default():

       self.session = tf.Session(graph=self.graph)

       self.session.run(init)

another thread:

   output = self.session.run([self.decoded[0]], test_feed)

tensorflow 读取模型（ckpt）结构,参数

from tensorflow.python importpywrap_tensorflow

def print_tensors_in_checkpoint_file(file_name):

   """Prints tensors in a checkpoint file.

   """

   try:

       reader = pywrap_tensorflow.NewCheckpointReader(file_name)

       var_to_shape_map = reader.get_variable_to_shape_map()

       for key in var_to_shape_map:

            print("tensor_name: ",key)

            print(reader.get_tensor(key))

       print(reader.debug_string().decode("utf-8"))

   except Exception as e:  # pylint:disable=broad-except

       print(str(e))

       if "corrupted compressed block contents" in str(e):

            print("It's likely that yourcheckpoint file has been compressed "

                  "with SNAPPY.")

调用：

print_tensors_in_checkpoint_file("****.ckpt")

RNN,LSTM,GRU正交初始化

def orthogonal_initializer(shape,dtype=tf.float32, *args, **kwargs):

 """Generates orthonormal matrices with random values.

 Orthonormal initialization is important for RNNs:

   http://arxiv.org/abs/1312.6120

   http://smerity.com/articles/2016/orthogonal_init.html

 For non-square shapes the returned matrix will be semi-orthonormal: ifthe

 number of columns exceeds the number of rows, then the rows areorthonormal

 vectors; but if the number of rows exceeds the number of columns, thenthe

 columns are orthonormal vectors.

 We use SVD decomposition to generate an orthonormal matrix with random

 values. The same way as it is done in the Lasagne library for Theano.Note

 that both u and v returned by the svd are orthogonal and random. We justneed

 to pick one with the right shape.

 Args:

   shape: a shape of the tensor matrix to initialize.

   dtype: a dtype of the initialized tensor.

   *args: not used.

   **kwargs: not used.

 Returns:

   An initialized tensor.

 """

 del args

 del kwargs

 flat_shape = (shape[0], np.prod(shape[1:]))

 w = np.random.randn(*flat_shape)

 u, _, v = np.linalg.svd(w, full_matrices=False)

 w = u if u.shape == flat_shape else v

 return tf.constant(w.reshape(shape), dtype=dtype)