Usage of Protocol Buffer
来源:互联网 发布:东方财富网 mac版本 编辑:程序博客网 时间:2024/05/22 02:29
Usage of Protocol Buffer
m
1 Introduction
Protocol buffers supports a language-neutral, platform-neutral,extensible way of serializing structured data for use in communicationsprotocols, data storage, and more.
Protocol buffers are a flexible, efficient, automated mechanism forserializing structured data – think XML, but smaller, faster, and simpler.However, protocol buffer will save data in binary mode, it’s not readable. XMLis readable. XML is a standard of W3C,protocol buffer is not a international standard.
Note: If you don’t care if the data is readable,you can use protocol buffer. The biggest merit is that you don’t need to writethe parser code and test it.
NOTE: If you want to start protocol bufferquickly, you only need to read part: language guide 3.1-3.3 and part 4:C++Tutorials.
2 What you need to do when usingprotocol buffer?
1) Download protocol buffer andinstall it;
2) Write the specification file todefine your structured information in .proto
files.
3) Compile the .proto
filesinto your language, C++, Java or Python.
4) Using the generated classes inyour code.
3 Language Guide (Writing .protocfile)
This guide describes how to use theprotocol buffer language to structure your protocol buffer data, including .proto
filesyntax and how to generate data access classes from your .proto
files.
3.1 Defining A Message Type
In protocol buffer, message type is like aclass or structure. It is the basic component of protoc file. You can defineyour message as follow:
message name{
Messagevar1; # field1
Messagevar2; # field2
……
}
Message Variable Format:
Field Rule Field Type var_name = Unique numbered Tag [default= 10]; //Comments
Field Rule defines how todeal with this field in protocol buffer.
* required: a well-formed message must have exactlyone of this field.
* optional: a well-formed message can have zero or oneof this field (but not more than one).
* repeated: this field can be repeated any number oftimes (including zero) in a well-formed message. The order of the repeatedvalues will be preserved.
Note: Required Is Forever. If you use required for one field, youdon’t have the change to modify the field any more. Thus pay more attention onrequired, using optional and repeated instead except that you are sure that thefield can’t be change any more.
Field Type defines the variable type. Field type is language-neutral, platform-neutral and is will be map to relatedactual variable type when you compile protoc file.
.proto Type
Notes
C++ Type
Java Type
double
double
double
float
float
float
int32
Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.
int32
int
int64
Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.
int64
long
uint32
Uses variable-length encoding.
uint32
int[1]
uint64
Uses variable-length encoding.
uint64
long[1]
sint32
Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.
int32
int
sint64
Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.
int64
long
fixed32
Always four bytes. More efficient than uint32 if values are often greater than 228.
uint32
int[1]
fixed64
Always eight bytes. More efficient than uint64 if values are often greater than 256.
uint64
long[1]
sfixed32
Always four bytes.
int32
int
sfixed64
Always eight bytes.
int64
long
bool
bool
boolean
string
A string must always contain UTF-8 encoded or 7-bit ASCII text.
string
String
bytes
May contain any arbitrary sequence of bytes.
string
ByteString
Unique Numbered Tag:Each field in the message definition has a uniquenumbered tag. These tags are used to identify your fields in the messagebinary format, and should not be changed once your message type is in use.
Note:
1) Tags with values in the range 1through 15 take one byte to encode. Tags in therange 16 through 2047 take two bytes. So youshould reserve the tags 1 through 15 for very frequently occurring messageelements and leave some room for frequently occurring elements that might beadded in the future.
2) Numbers19000 though 19999 are reserved for the Protocol Buffersimplementation, not use them.
Protocol buffer alsosupport Enum, Nested message, Package, import and Service (RPC--Remote Procedure Call). We will learn them from example.
3.2 Example
package EXAM; // define apackage, like namespace
import "myproject/other_protos.proto"; // import otherprotoc file, like include
message SearchRequest { //define a message
required string query = 1; //define message fields
optional int32 page_number = 2;
optional int32 result_per_page = 3 [default = 10];
enum Corpus { // define an enum a
UNIVERSAL = 0;
WEB = 1;
IMAGES = 2;
LOCAL = 3;
NEWS = 4;
PRODUCTS = 5;
VIDEO = 6;
}
optional Corpuscorpus = 4 [default = UNIVERSAL]; // use an option value
message Result { // nested message
required string url = 1;
optional string title = 2;
repeated string snippets =3;
}
repeated Resultresult = 5;
}
message other{ // define multi messages in samefile
requiredstring url = 1;
…..
}
========================================================
3.3 Generating Your Classes
To generate the Java, Python, or C++ codeyou need to work with the message types defined in a .proto
file,you need to run the protocol buffer compiler protoc
on the .proto
.Example:
protoc --proto_path=IMPORT_PATH --cpp_out=DST_DIR --java_out=DST_DIR --python_out=DST_DIR path/to/file.proto
This command will compile protoc file intoC++/Java/Python code.
Using “protoc –help” for detail.
3.4 Extensions
Extensions let you declare that a range offield numbers, which are available for third-partyextensions, in a message. Other peoplecan then declare new fields for your message type with those numeric tags in their own .proto
files without having to edit the original file. Let's look at an example:
message Foo {
// ...
extensions 100 to 199;
}
Other users can now add new fields to Foo
in theirown .proto
files that import your .proto
, using tagswithin your specified range – for example:
extend Foo {
optional int32 bar = 126;
}
However, the way you access extensionfields in your application code is slightly different to accessing regularfields – your generated data access code has special accessors for working withextensions. So, for example, here's how you set the value of bar in C++:
Foo foo;
foo.SetExtension(bar, 15);
3.5 Rules for Updating A Message Type
l Don't change the numeric tagsfor any existing fields.
l Any new fields that you addshould be optional or repeated.
l Non-required fields can be removed,as long as the tag number is not used again in your updated message type.
l A non-required field can beconverted to an extension and vice versa, as long as the type and number staythe same.
l int32, uint32, int64, uint64,and bool are all compatible – this means you can change a field from one ofthese types to another without breaking forwards- or backwards-compatibility.
l sint32 and sint64 arecompatible with each other but are not compatible with the other integer types.
l string and bytes are compatibleas long as the bytes are valid UTF-8.
l Embedded messages arecompatible with bytes if the bytes contain an encoded version of the message.
l fixed32 is compatible withsfixed32, and fixed64 with sfixed64.
3.6 FAQ
Ø Can I change or the generated code?
Don’t to that! Protocol buffer classes are basically dumb data holders (likestructs in C++). If you want to add richer behavior to a generated class, thebest way to do this is to wrap the generated protocol buffer class in anapplication-specific class.You should never add behavior to thegenerated classes by inheriting from them.This will break internal mechanisms and is not good object-oriented practiceanyway.
Ø How to write multi messages into one file (buffer)?
If you want to write multiple messages to a single file or stream,it is up to you to keep track of where one message ends and the next begins. Inother word, you need to write/read the message serialization string to/fromfile correctly.
Ø Can protocol buffer deal with Large Data Sets?
Protocol Buffers are not designed to handle large messages. As ageneral rule of thumb, if you are dealing in messages larger than a megabyteeach, it may be time to consider an alternate strategy.
Ø How to optimize protocol generated code?
1) Using option, Such as “option optimize_for= LITE_RUNTIME
/CODE_SIZE;”. 2) Reuse message objects when possible.
4 C++ Tutorials
Common API
Standard Message Methods
Each message class also contains a numberof other methods that let you check or manipulate the entire message,including:
* boolIsInitialized() const;: checks if allthe required fields have been set.
* string DebugString() const;: returns ahuman-readable representation of the message, particularly useful fordebugging.
* void CopyFrom(const Person& from);: overwritesthe message with the given message's values.
* void Clear();: clears all theelements back to the empty state.
Parsing and Serialization
Each protocol buffer class has methods forwriting and reading messages of your chosen type using the protocol bufferbinary format. These include:
* bool SerializeToString(string* output) const;: serializes the message and stores the bytes in the given string.Note that the bytes are binary, not text; we only use the string class as aconvenient container.
* bool ParseFromString(const string& data);: parses a message from the given string.
* bool SerializeToOstream(ostream* output) const;: writes the message to the given C++ ostream.
* bool ParseFromIstream(istream* input);: parses a message from the given C++ istream.
Example
The example code isincluded in the source code package, under the "examples" directory.Study the example!!!
Refer to http://code.google.com/intl/zh-CN/apis/protocolbuffers/docs/cpptutorial.html
NOTE: For each field, get_field() andset_field() interfaces will be generated. However,these interfaces only support lowercase field regardless of Field or field isused in protoc file.
- Usage of Protocol Buffer
- protocol buffer
- protocol buffer
- Protocol Buffer
- protocol Buffer
- protocol buffer
- Protocol Buffer
- Protocol Buffer
- Protocol Buffer
- Protocol Buffer
- Protocol Buffer
- Protocol Buffer
- Protocol Buffer
- Protocol Buffer
- Protocol Buffer
- Protocol Buffer
- (OS 10048)Only one usage of each socket address (protocol/network address/port) is normally permitte
- Google Protocol Buffer
- Linux Bash Shell学习(九):流程控制——if/else
- iSCSI Initiator与 Target 的安装、配置与连接过程备忘
- 颜小茕..你来对地方了吧..哈!
- ArcEngine唯一值渲染UniqueValueRender图层
- lucene-搜索过滤
- Usage of Protocol Buffer
- 美著名IT网站评出互联网史上最成功的七大骗局
- 求助
- lucene-SpanNotQuery和SpanOrQuery交迭与全局跨度
- 使用VS的安装项目的总结
- Windows Mail(Outlook Express)添加图片签名
- lucene-SpanFirstQuery 和SpanNearQuery 跨度查询
- 以iPhone的名义:一个开发培训基地的草根成长
- Singleton 进一步学习