Fine-tuning a model from an existing checkpoint Rather than training from scratch, we'll often want
来源:互联网 发布:php更改上传文件大小 编辑:程序博客网 时间:2024/06/05 17:32
Fine-tuning a model from an existing checkpoint
Rather than training from scratch, we'll often want to start from a pre-trainedmodel and fine-tune it.To indicate a checkpoint from which to fine-tune, we'll call training withthe --checkpoint_path
flag and assign it an absolute path to a checkpointfile.
When fine-tuning a model, we need to be careful about restoring checkpointweights. In particular, when we fine-tune a model on a new task with a differentnumber of output labels, we wont be able restore the final logits (classifier)layer. For this, we'll use the --checkpoint_exclude_scopes
flag. This flaghinders certain variables from being loaded. When fine-tuning on aclassification task using a different number of classes than the trained model,the new model will have a final 'logits' layer whose dimensions differ from thepre-trained model. For example, if fine-tuning an ImageNet-trained model onFlowers, the pre-trained logits layer will have dimensions [2048 x 1001]
butour new logits layer will have dimensions [2048 x 5]
. Consequently, thisflag indicates to TF-Slim to avoid loading these weights from the checkpoint.
Keep in mind that warm-starting from a checkpoint affects the model's weightsonly during the initialization of the model. Once a model has started training,a new checkpoint will be created in${TRAIN_DIR}
. If the fine-tuningtraining is stopped and restarted, this new checkpoint will be the one fromwhich weights are restored and not the ${checkpoint_path}$
. Consequently,the flags--checkpoint_path
and --checkpoint_exclude_scopes
are only usedduring the 0-
th global step (model initialization). Typically for fine-tuningone only want train a sub-set of layers, so the flag--trainable_scopes
allowsto specify which subsets of layers should trained, the rest would remain frozen.
Below we give an example offine-tuning inception-v3 on flowers,inception_v3 was trained on ImageNet with 1000 class labels, but the flowersdataset only have 5 classes. Since the dataset is quite small we will only trainthe new layers.
$ DATASET_DIR=/tmp/flowers$ TRAIN_DIR=/tmp/flowers-models/inception_v3$ CHECKPOINT_PATH=/tmp/my_checkpoints/inception_v3.ckpt$ python train_image_classifier.py \ --train_dir=${TRAIN_DIR} \ --dataset_dir=${DATASET_DIR} \ --dataset_name=flowers \ --dataset_split_name=train \ --model_name=inception_v3 \ --checkpoint_path=${CHECKPOINT_PATH} \ --checkpoint_exclude_scopes=InceptionV3/Logits,InceptionV3/AuxLogits \ --trainable_scopes=InceptionV3/Logits,InceptionV3/AuxLogits
Evaluating performance of a model
To evaluate the performance of a model (whether pretrained or your own),you can use the eval_image_classifier.py script, as shown below.
Below we give an example of downloading the pretrained inception model andevaluating it on the imagenet dataset.
CHECKPOINT_FILE = ${CHECKPOINT_DIR}/inception_v3.ckpt # Example$ python eval_image_classifier.py \ --alsologtostderr \ --checkpoint_path=${CHECKPOINT_FILE} \ --dataset_dir=${DATASET_DIR} \ --dataset_name=imagenet \ --dataset_split_name=validation \ --model_name=inception_v3
See the evaluation module examplefor an example of how to evaluate a model at multiple checkpoints during or after the training.
Exporting the Inference Graph
Saves out a GraphDef containing the architecture of the model.
To use it with a model name defined by slim, run:
$ python export_inference_graph.py \ --alsologtostderr \ --model_name=inception_v3 \ --output_file=/tmp/inception_v3_inf_graph.pb$ python export_inference_graph.py \ --alsologtostderr \ --model_name=mobilenet_v1 \ --image_size=224 \ --output_file=/tmp/mobilenet_v1_224.pb
Freezing the exported Graph
If you then want to use the resulting model with your own or pretrainedcheckpoints as part of a mobile model, you can run freeze_graph to get a graphdef with the variables inlined as constants using:
bazel build tensorflow/python/tools:freeze_graphbazel-bin/tensorflow/python/tools/freeze_graph \ --input_graph=/tmp/inception_v3_inf_graph.pb \ --input_checkpoint=/tmp/checkpoints/inception_v3.ckpt \ --input_binary=true --output_graph=/tmp/frozen_inception_v3.pb \ --output_node_names=InceptionV3/Predictions/Reshape_1
The output node names will vary depending on the model, but you can inspect andestimate them using the summarize_graph tool:
bazel build tensorflow/tools/graph_transforms:summarize_graphbazel-bin/tensorflow/tools/graph_transforms/summarize_graph \ --in_graph=/tmp/inception_v3_inf_graph.pb
Run label image in C++
To run the resulting graph in C++, you can look at the label_image sample code:
bazel build tensorflow/examples/label_image:label_imagebazel-bin/tensorflow/examples/label_image/label_image \ --image=${HOME}/Pictures/flowers.jpg \ --input_layer=input \ --output_layer=InceptionV3/Predictions/Reshape_1 \ --graph=/tmp/frozen_inception_v3.pb \ --labels=/tmp/imagenet_slim_labels.txt \ --input_mean=0 \ --input_std=255
- Fine-tuning a model from an existing checkpoint Rather than training from scratch, we'll often want
- BDNtv: Deriving a model from an existing database with ECO II in Delphi 2005
- Generating EF Code First model classes from an existing database
- Building a Visual Robot Model with URDF from Scratch
- what we want from the game?
- A Groovy DSL from scratch
- Building a PyCommand from scratch
- Install Kubernetes From a Scratch
- Swift from Scratch: An Introduction to Functions
- Implementing a Neural Network from Scratch in Python – An Introduction(通过python中的Scratch来实现神经网络--简介)
- How to create an EMF model from a Java application ?
- Implementing a Neural Network from Scratch in Python – An Introduction
- Script to Generate a CREATE DATABASE command from an existing database.
- Create a new repository on the command line/Push an existing repository from the command line
- XenServer 6.5实战系列之八:Creating a VM Template from an Existing VM
- How to create a server from scratch
- start a android App from scratch
- Creating a class from scratch with Soot
- HDU2022海选女主角
- times函数
- Java The Complete Reference, 8th Edition.pdf 英文原版 免费下载
- Java高级之线程(二)
- Python 3 Object-Oriented Programming, Second Edition.pdf 英文原版 免费下载
- Fine-tuning a model from an existing checkpoint Rather than training from scratch, we'll often want
- hashmap的底层实现
- 蓝桥杯选拔赛题解
- 动态规划解343. Integer Break
- linux proc_root_init
- Springboot中使用数据库配置Quartz定时任务
- 小程序
- Expert SQL Server In-Memory OLTP, 2nd Edition.pdf 英文原版 免费下载
- Java Develop——基于 Redis 的分布式锁