如何在SAS中重新构建限价指令簿（Limit Order Book）：读取NASDAQ Total View 4.1数据

不论是开发高频交易策略还是研究高频交易对于市场微观结构的影响都需要用到逐个指令的数据，而目前所有交易所提供的这类数据中以NASDAQ Total View数据最具代表性，目前最新的数据格式是4.1。有了这些数据之后就需要重新构建限价指令簿（Limit Order Book），为了达成这一目标需要克服三个难点：

1）NASDAQ Total View 4.1数据是二进制文件，而且数据是以message的形式一条一条记录，每个message代表的就是一次提单或撤单或成交。读取这样的数据会是件比较头疼的事。

2）只有提单一类的message才会包含股票的Ticker，也就是告诉你这个单子是关于哪个股票的，同时会给你一个Reference Number，但是撤单和成交这些只对应于Reference Number，不会声明是哪只股票。

3）有了所有的message，并且知道对应的是哪个股票之后就可以开始构建限价指令簿。

目前难点1）、3）已经可以很好的解决，但是2）还较麻烦，会通过几篇博客再介绍，这里先谈第一步。

NASDAQ Total View 4.1数据是以天为单位，每天所有股票的单子都放在同一个二进制文件中，最少也要1GB，多的可以有7、8GB，对于这么大的数据，硬件上最大的制约就是硬盘读写速度。SAS的一大特点就是Data步处理数据时会边从硬盘读数据边处理，处理完就写到硬盘上，读一天的数据少则半个小时很正常。

剩下的就是数据格式，NASDAQ Total View 4.1数据是以message的形式记录，message会分不同种类，例如：提单、撤单、成交、换单。但是每个message的开头都是都是同一格式，即表示本message的长度，接着会有一个字母表示本message的种类，之后就是其他。

具体的代码如下，当然这是国外某人写的（http://charlesmartineau.com/wp-content/uploads/2012/12/itch_4.1_format.txt ），如要节省空间还可以再改

data want;  infile "C:\S120612-v41.txt" RECFM=N;  input width s370fpib2. msg_type $1.;  select(msg_type);       /* Time Stamp - Seconds */        when('T') do;          input seconds s370fpib4.;        end;       /* System Event */        when('S') do;          input nanoseconds s370fpib4.                  event $1.;        end;       /* Stock Related Messages */        when('R') do;          input nanoseconds s370fpib4.                  stock $8.                  market_category $1.                  financial_status $1.                  round_lot_size s370fpib4.                  round_lots_only $1.;        end;       /* Stock Trading Action */        when('H') do;          input nanoseconds s370fpib4.                  stock $8.                  trading_state $1.                  reserved $1.                  reason $4.;        end;   /* Reg SHO Short Sale Price TEst Restricted Indicator */    when ('Y') do;  input nanoseconds s370fpib4.  stock $8.  Reg_SHO_action $1.;    end;       /* Market Participant Position */        when('L') do;          input nanoseconds s370fpib4.                  mpid $4.                  stock $8.                  primary_market_maker $1.                  market_maker_mode $1.                  market_participant_state $1.;        end;       /* Add Order No MPID Attribution */        when('A') do;          input nanoseconds s370fpib4.                  order_reference_number s370fpib8.                  buy_sell_indicator $1.                  shares s370fpib4.                  stock $8.                  price s370fpib4.;        end;       /* Add Order with MPID Attribution */        when('F') do;          input nanoseconds s370fpib4.                  order_reference_number s370fpib8.                  buy_sell_indicator $1.                  shares s370fpib4.                  stock $8.                  price s370fpib4.                  attribution $4.;        end;       /* Order Executed */        when('E') do;          input nanoseconds s370fpib4.                  order_reference_number s370fpib8.                  executed_shares s370fpib4.                  match_number s370fpib8.;        end;       /* Order Executed with Price */        when('C') do;          input nanoseconds s370fpib4.                  order_reference_number s370fpib8.                  executed_shares s370fpib4.                  match_number s370fpib8.                  printable $1.                  execution_price s370fpib4.;        end;       /* Order Cancel */        when('X') do;          input nanoseconds s370fpib4.                  order_reference_number s370fpib8.                  canceled_shares s370fpib4.;        end;       /* Order Delete */        when('D') do;          input nanoseconds s370fpib4.                  order_reference_number s370fpib8.;        end;       /* Order Replace */        when('U') do;          input nanoseconds s370fpib4.                  order_reference_number s370fpib8.                  new_order_reference_number s370fpib8.                  shares s370fpib4.                  price s370fpib4.;        end;       /* Trade Message Non-Cross */        when('P') do;          input nanoseconds s370fpib4.                  order_reference_number s370fpib8.                  buy_sell_indicator $1.                  shares s370fpib4.                  stock $8.                  price s370fpib4.                  match_number s370fpib8.;        end;       /* Cross Trade Message */        when('Q') do;          input nanoseconds s370fpib4.                  shares_big s370fpib8.                  stock $8.                  cross_price s370fpib4.                  match_number s370fpib8.                  cross_type $1.;        end;       /* Broken Trade Order Execution */        when('B') do;          input nanoseconds s370fpib4.                  match_number s370fpib8.;        end;       /* Net Order Imbalance msg_type = 'I'*/        when ('I') do;          input nanoseconds s370fpib4.                  paired_shares s370fpib8.                  imbalance_shares s370fpib8.                  imbalance_direction $1.                  stock $8.                  fair_price s370fpib4.                  near_price s370fpib4.                  current_reference_price s370fpib4.                  cross_type $1.                  price_variation_indicator $1.;        end;   /* Retail Price Improvement Indicator (RPII) - letter 'G' */    otherwise do;  input nanoseconds s370fpib4.  stock $8.  interest_flag $1.;end;  end;run ;