tfRecord写入

本文为google inception-v3范例代码中图像预处理相关代码的阅读笔记
主要包括图像读取以及如何将图片写入tfRecord

文件读写相关

从txt_file_name中读取txt文件 返回一个list，list的每个元素为txt中的行元素

tf.gfile.FastGFile(txt_file_name,'r').readlines()

从file_path代表的正则表达式中读取文件，返回一个list，list中的每个元素代表一个文件名路径
使用tensorflow读取图片

tf.gfile.Glob(file_path)

---

tensorflow简单图像处理

(1) 创建tensorflow图像处理对象

coder= ImageCoder()

ImageCoder()类的定义：

class ImageCoder(object):  """Helper class that provides TensorFlow image coding utilities."""  def __init__(self):    # Create a single Session to run all image coding calls.    self._sess = tf.Session()    # Initializes function that converts PNG to JPEG data.    self._png_data = tf.placeholder(dtype=tf.string)    image = tf.image.decode_png(self._png_data, channels=3)    self._png_to_jpeg = tf.image.encode_jpeg(image, format='rgb', quality=100)    # Initializes function that decodes RGB JPEG data.    self._decode_jpeg_data = tf.placeholder(dtype=tf.string)    self._decode_jpeg = tf.image.decode_jpeg(self._decode_jpeg_data, channels=3)  def png_to_jpeg(self, image_data):    return self._sess.run(self._png_to_jpeg,                          feed_dict={self._png_data: image_data})  def decode_jpeg(self, image_data):    image = self._sess.run(self._decode_jpeg,                           feed_dict={self._decode_jpeg_data: image_data})    assert len(image.shape) == 3    assert image.shape[2] == 3    return image

(2)读取图像

with tf.gfile.FastGFile(filename,'rb') as f:    image_data = f.read()

tfrecord相关函数

（1）线程控制相关
线程监控器，用来监控是否所有线程已经结束

coord = tf.train.Coordinator()

usage:

–1创建线程

coord=Coordinator()

–2发起一系列线程，将coordinator传递给每个线程

....start thread1...(coord,...)...start thread N...(coord,...)

–3等待线程执行完毕

coord.join(threads)

---

写tfRecord的流程：

–1 创建一个写tfRecord的对象：

writer = tf.python_io.TFRecordWriter(dst_file_name)

–2 读取图片，并将其转换成二进制

image_buffer, height, width = _process_image(filename, coder)

_process_image源码

def _process_image(filename, coder):  """Process a single image file.  Args:    filename: string, path to an image file e.g., '/path/to/example.JPG'.    coder: instance of ImageCoder to provide TensorFlow image coding utils.  Returns:    image_buffer: string, JPEG encoding of RGB image.    height: integer, image height in pixels.    width: integer, image width in pixels.  """  # Read the image file.  with tf.gfile.FastGFile(filename, 'rb') as f:    image_data = f.read()  # Convert any PNG to JPEG's for consistency.  if _is_png(filename):    print('Converting PNG to JPEG for %s' % filename)    image_data = coder.png_to_jpeg(image_data)  # Decode the RGB JPEG.  image = coder.decode_jpeg(image_data)  # Check that image converted to RGB  assert len(image.shape) == 3  height = image.shape[0]  width = image.shape[1]  assert image.shape[2] == 3  return image_data, height, width

–3将图片转化为example。example为一张图片在tfRecord存储的一个结构化单元，包含了图片内容和基本信息

  example = _convert_to_example(filename, image_buffer, label,text, height, width)

_convert_to_example定义：

def _convert_to_example(filename, image_buffer, label, text, height, width):  """Build an Example proto for an example.  Args:    filename: string, path to an image file, e.g., '/path/to/example.JPG'    image_buffer: string, JPEG encoding of RGB image    label: integer, identifier for the ground truth for the network    text: string, unique human-readable, e.g. 'dog'    height: integer, image height in pixels    width: integer, image width in pixels  Returns:    Example proto  """  colorspace = 'RGB'  channels = 3  image_format = 'JPEG'  example = tf.train.Example(features=tf.train.Features(feature={      'image/height': _int64_feature(height),      'image/width': _int64_feature(width),      'image/colorspace': _bytes_feature(tf.compat.as_bytes(colorspace)),      'image/channels': _int64_feature(channels),      'image/class/label': _int64_feature(label),      'image/class/text': _bytes_feature(tf.compat.as_bytes(text)),      'image/format': _bytes_feature(tf.compat.as_bytes(image_format)),      'image/filename': _bytes_feature(tf.compat.as_bytes(os.path.basename(filename))),      'image/encoded': _bytes_feature(tf.compat.as_bytes(image_buffer))}))  return example

其中子函数定义：

def _int64_feature(value):  """Wrapper for inserting int64 features into Example proto."""  if not isinstance(value, list):    value = [value]  return tf.train.Feature(int64_list=tf.train.Int64List(value=value))def _bytes_feature(value):  """Wrapper for inserting bytes features into Example proto."""  return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

–4将一个example写入tfRecord文件，注意需要将example转换为字符串流：

writer.write(example.SerializeToString())

–5写完后不要忘记关闭writer对象～

writer.close()