Stanford NER CRF FAQ

原网页http://nlp.stanford.edu/software/crf-faq.html

• 1.我能够训练自己的NER模型么？
• 2.我可以使用尽量少的内存来训练NER模型么？
• 3.我如何从多个文件中训练一个模型？
• 4.在程序中使用CRFClassifier，API是什么?
• 5.我可以设置斯坦福NER系统，允许单个jar部署，而不是必须从分离的多个文件加载NER模型？
• 6.对于我们的web 5.0系统，我能够运行Stanford NER作为一个server/service/servlet？
• 7.有哪些选项可用于格式化分类器的输出？
• 8.为什么我无法重现您的CoNLL 2003系统的结果？ 或者：我如何使用CoNLL 2003在英语上获得更好的性能？
• 9.如何让德语NER工作正常？ 我对字符编码理解不深。
• 10.什么是分类器的渐进内存增长？
• 11.现有的模型可以扩展么？
• 12.你可以发布用于构建公开发布的模型的数据么？
• 13.NER结果是确定性的吗？ 为什么同一数据的结果会发生变化？
• 14.如何NER标记已分词的文本？
• 15.什么选项可用于文本输入处理？
• 16.NER是否使用词性标签？
• 17.我如何在Stanford NER的帮助下使用gazette？
• 18.斯坦福NER可以与Solr集成么？

这里仅仅先只对第一个大致翻译
你应该至少浏览CRFClassifier 和 NERFeatureFactory类的javaDoc。
训练数据应该是使用tab键分隔的列，至少要有两列，单词一列，单词的类别标签一列。
新建jane-austen-emma-ch1.txt文件（将下面内容粘贴进去），或者直接下载从http://nlp.stanford.edu/software/ner-example/jane-austen-emma-ch1.txt

CHAPTER IEmma Woodhouse handsome clever and rich with a comfortable homeand happy disposition seemed to unite some of the best blessingsof existence and had lived nearly twenty one years in the worldwith very little to distress or vex her She was the youngest of the two daughters of a most affectionateindulgent father and had in consequence of her sister's marriagebeen mistress of his house from a very early period  Her motherhad died too long ago for her to have more than an indistinctremembrance of her caresses and her place had been suppliedby an excellent woman as governess who had fallen little shortof a mother in affection Sixteen years had Miss Taylor been in Mr Woodhouse's familyless as a governess than a friend very fond of both daughtersbut particularly of Emma  Between them it was more the intimacyof sisters  Even before Miss Taylor had ceased to hold the nominaloffice of governess the mildness of her temper had hardly allowedher to impose any restraint and the shadow of authority beingnow long passed away they had been living together as friend andfriend very mutually attached and Emma doing just what she likedhighly esteeming Miss Taylor's judgment but directed chiefly byher own The real evils indeed of Emma's situation were the power of havingrather too much her own way and a disposition to think a littletoo well of herself these were the disadvantages which threatenedalloy to her many enjoyments  The danger however was at presentso unperceived that they did not by any means rank as misfortuneswith her Sorrow came  a gentle sorrow  but not at all in the shape of anydisagreeable consciousness  Miss Taylor married  It was MissTaylor's loss which first brought grief  It was on the wedding dayof this beloved friend that Emma first sat in mournful thoughtof any continuance  The wedding over and the bride people goneher father and herself were left to dine together with no prospectof a third to cheer a long evening  Her father composed himselfto sleep after dinner as usual and she had then only to sitand think of what she had lost The event had every promise of happiness for her friend  Mr Westonwas a man of unexceptionable character easy fortune suitable ageand pleasant manners and there was some satisfaction in consideringwith what self denying generous friendship she had always wishedand promoted the match but it was a black morning's work for her The want of Miss Taylor would be felt every hour of every day She recalled her past kindness  the kindness the affection of sixteenyears  how she had taught and how she had played with her from fiveyears old  how she had devoted all her powers to attach and amuseher in health  and how nursed her through the various illnessesof childhood  A large debt of gratitude was owing here but theintercourse of the last seven years the equal footing and perfectunreserve which had soon followed Isabella's marriage on theirbeing left to each other was yet a dearer tenderer recollection She had been a friend and companion such as few possessed intelligentwell informed useful gentle knowing all the ways of the familyinterested in all its concerns and peculiarly interested in herselfin every pleasure every scheme of hers  one to whom she could speakevery thought as it arose and who had such an affection for heras could never find fault How was she to bear the change  It was true that her friend wasgoing only half a mile from them but Emma was aware that great mustbe the difference between a Mrs Weston only half a mile from themand a Miss Taylor in the house and with all her advantagesnatural and domestic she was now in great danger of sufferingfrom intellectual solitude  She dearly loved her father but hewas no companion for her  He could not meet her in conversationrational or playful The evil of the actual disparity in their ages and Mr Woodhouse hadnot married early was much increased by his constitution and habitsfor having been a valetudinarian all his life without activityof mind or body he was a much older man in ways than in yearsand though everywhere beloved for the friendliness of his heartand his amiable temper his talents could not have recommended himat any time Her sister though comparatively but little removed by matrimonybeing settled in London only sixteen miles off was much beyondher daily reach and many a long October and November evening mustbe struggled through at Hartfield before Christmas brought the nextvisit from Isabella and her husband and their little childrento fill the house and give her pleasant society again Highbury the large and populous village almost amounting to a townto which Hartfield in spite of its separate lawn and shrubberiesand name did really belong afforded her no equals  The Woodhouseswere first in consequence there  All looked up to them  She hadmany acquaintance in the place for her father was universally civilbut not one among them who could be accepted in lieu of MissTaylor for even half a day  It was a melancholy change and Emmacould not but sigh over it and wish for impossible thingstill her father awoke and made it necessary to be cheerful His spirits required support  He was a nervous man easily depressedfond of every body that he was used to and hating to part with themhating change of every kind  Matrimony as the origin of changewas always disagreeable and he was by no means yet reconciledto his own daughter's marrying nor could ever speak of her butwith compassion though it had been entirely a match of affectionwhen he was now obliged to part with Miss Taylor too and fromhis habits of gentle selfishness and of being never able tosuppose that other people could feel differently from himselfhe was very much disposed to think Miss Taylor had done as sada thing for herself as for them and would have been a great dealhappier if she had spent all the rest of her life at Hartfield Emma smiled and chatted as cheerfully as she could to keep himfrom such thoughts but when tea came it was impossible for himnot to say exactly as he had said at dinnerPoor Miss Taylor  I wish she were here again  What a pity itis that Mr Weston ever thought of herI cannot agree with you papa you know I cannot  Mr Weston is sucha good humoured pleasant excellent man that he thoroughly deservesa good wife  and you would not have had Miss Taylor live with usfor ever and bear all my odd humours when she might have a house of her ownA house of her own  But where is the advantage of a house of her ownThis is three times as large  And you have never any odd humoursmy dearHow often we shall be going to see them and they coming to seeus  We shall be always meeting We must begin we must go and paywedding visit very soonMy dear how am I to get so far Randalls is such a distance I could not walk half so farNo papa nobody thought of your walking  We must go in the carriageto be sureThe carriage But James will not like to put the horses to forsuch a little way  and where are the poor horses to be while weare paying our visitThey are to be put into Mr Weston's stable papa  You know wehave settled all that already  We talked it all over with Mr Westonlast night  And as for James you may be very sure he will always likegoing to Randalls because of his daughter's being housemaid there I only doubt whether he will ever take us anywhere else  That wasyour doing papa  You got Hannah that good place  Nobody thoughtof Hannah till you mentioned her  James is so obliged to youI am very glad I did think of her  It was very lucky for I wouldnot have had poor James think himself slighted upon any accountand I am sure she will make a very good servant she is a civilpretty spoken girl I have a great opinion of her  Whenever I see hershe always curtseys and asks me how I do in a very pretty mannerand when you have had her here to do needlework I observe shealways turns the lock of the door the right way and never bangs it I am sure she will be an excellent servant and it will be a greatcomfort to poor Miss Taylor to have somebody about her that she isused to see  Whenever James goes over to see his daughter you knowshe will be hearing of us  He will be able to tell her how weall areEmma spared no exertions to maintain this happier flow of ideasand hoped by the help of backgammon to get her father tolerablythrough the evening and be attacked by no regrets but her own The backgammon table was placed but a visitor immediately afterwardswalked in and made it unnecessary Mr Knightley a sensible man about seven or eight and thirty was notonly a very old and intimate friend of the family but particularlyconnected with it as the elder brother of Isabella's husband He lived about a mile from Highbury was a frequent visitorand always welcome and at this time more welcome than usualas coming directly from their mutual connexions in London  He hadreturned to a late dinner after some days absence and now walkedup to Hartfield to say that all were well in Brunswick Square It was a happy circumstance and animated Mr Woodhouse for some time Mr Knightley had a cheerful manner which always did him goodand his many inquiries after poor Isabella and her children wereanswered most satisfactorily  When this was over Mr Woodhousegratefully observed It is very kind of you Mr Knightley to comeout at this late hour to call upon us  I am afraid you must havehad a shocking walkNot at all sir  It is a beautiful moonlight night and so mildthat I must draw back from your great fireBut you must have found it very damp and dirty  I wish you maynot catch coldDirty sir Look at my shoes  Not a speck on themWell that is quite surprising for we have had a vast dealof rain here  It rained dreadfully hard for half an hourwhile we were at breakfast  I wanted them to put off the weddingBy the bye  I have not wished you joy  Being pretty well awareof what sort of joy you must both be feeling I have been in no hurrywith my congratulations but I hope it all went off tolerably well How did you all behave Who cried mostAh poor Miss Taylor Tis a sad businessPoor Mr and Miss Woodhouse if you please but I cannot possiblysay poor Miss Taylor I have a great regard for you and Emmabut when it comes to the question of dependence or independence  Atany rate it must be better to have only one to please than twoEspecially when one of those two is such a fanciful troublesome creaturesaid Emma playfully  That is what you have in your headI know  and what you would certainly say if my father were not byI believe it is very true my dear indeed said Mr Woodhousewith a sigh  I am afraid I am sometimes very fanciful and troublesomeMy dearest papa You do not think I could mean you or supposeMr Knightley to mean you  What a horrible idea Oh no I meantonly myself  Mr Knightley loves to find fault with me you know  in a joke  it is all a joke  We always say what we like to one anotherMr Knightley in fact was one of the few people who could seefaults in Emma Woodhouse and the only one who ever told her of themand though this was not particularly agreeable to Emma herselfshe knew it would be so much less so to her father that she wouldnot have him really suspect such a circumstance as her not beingthought perfect by every body Emma knows I never flatter her said Mr Knightley but Imeant no reflection on any body  Miss Taylor has been usedto have two persons to please she will now have but one The chances are that she must be a gainerWell said Emma willing to let it pass  you want to hearabout the wedding and I shall be happy to tell you for we allbehaved charmingly  Every body was punctual every body in theirbest looks not a tear and hardly a long face to be seen  Oh nowe all felt that we were going to be only half a mile apartand were sure of meeting every dayDear Emma bears every thing so well said her father But Mr Knightley she is really very sorry to lose poor Miss Taylorand I am sure she will miss her more than she thinks forEmma turned away her head divided between tears and smiles It is impossible that Emma should not miss such a companionsaid Mr Knightley  We should not like her so well as we do sirif we could suppose it but she knows how much the marriage is toMiss Taylor's advantage she knows how very acceptable it must beat Miss Taylor's time of life to be settled in a home of her ownand how important to her to be secure of a comfortable provisionand therefore cannot allow herself to feel so much pain as pleasure Every friend of Miss Taylor must be glad to have her so happilymarriedAnd you have forgotten one matter of joy to me said Emmaand a very considerable one  that I made the match myself I made the match you know four years ago and to have it take placeand be proved in the right when so many people said Mr Weston wouldnever marry again may comfort me for any thingMr Knightley shook his head at her  Her father fondly repliedAh my dear I wish you would not make matches and foretell thingsfor whatever you say always comes to pass  Pray do not make anymore matchesI promise you to make none for myself papa but I must indeedfor other people  It is the greatest amusement in the world Andafter such success you know  Every body said that Mr Weston wouldnever marry again  Oh dear no Mr Weston who had been a widowerso long and who seemed so perfectly comfortable without a wifeso constantly occupied either in his business in town or among hisfriends here always acceptable wherever he went always cheerful  Mr Weston need not spend a single evening in the year alone if he didnot like it  Oh no Mr Weston certainly would never marry again Some people even talked of a promise to his wife on her deathbedand others of the son and the uncle not letting him  All mannerof solemn nonsense was talked on the subject but I believed noneof it Ever since the day  about four years ago  that Miss Taylor and Imet with him in Broadway Lane when because it began to drizzlehe darted away with so much gallantry and borrowed two umbrellasfor us from Farmer Mitchell's I made up my mind on the subject I planned the match from that hour and when such success has blessedme in this instance dear papa you cannot think that I shall leaveoff match makingI do not understand what you mean by success said Mr Knightley Success supposes endeavour  Your time has been properly anddelicately spent if you have been endeavouring for the last fouryears to bring about this marriage  A worthy employment for a younglady's mind But if which I rather imagine your making the matchas you call it means only your planning it your saying to yourselfone idle day I think it would be a very good thing for Miss Taylorif Mr Weston were to marry her and saying it again to yourselfevery now and then afterwards why do you talk of success Whereis your merit What are you proud of You made a lucky guessand that is all that can be saidAnd have you never known the pleasure and triumph of a lucky guess  I pity you  I thought you cleverer  for depend upon it a luckyguess is never merely luck  There is always some talent in it And as to my poor word success which you quarrel with I do notknow that I am so entirely without any claim to it  You have drawntwo pretty pictures but I think there may be a third  a somethingbetween the do nothing and the do all If I had not promoted Mr Weston'svisits here and given many little encouragements and smoothedmany little matters it might not have come to any thing after all I think you must know Hartfield enough to comprehend thatA straightforward open hearted man like Weston and a rationalunaffected woman like Miss Taylor may be safely left to manage theirown concerns  You are more likely to have done harm to yourselfthan good to them by interferenceEmma never thinks of herself if she can do good to othersrejoined Mr Woodhouse understanding but in part  But my dearpray do not make any more matches they are silly things and break upone's family circle grievouslyOnly one more papa only for Mr Elton  Poor Mr Elton Youlike Mr Elton papa  I must look about for a wife for him There is nobody in Highbury who deserves him  and he has beenhere a whole year and has fitted up his house so comfortablythat it would be a shame to have him single any longer  and I thoughtwhen he was joining their hands to day he looked so very much as ifhe would like to have the same kind office done for him I thinkvery well of Mr Elton and this is the only way I have of doinghim a serviceMr Elton is a very pretty young man to be sure and a verygood young man and I have a great regard for him  But if youwant to shew him any attention my dear ask him to comeand dine with us some day  That will be a much better thing I dare say Mr Knightley will be so kind as to meet himWith a great deal of pleasure sir at any time said Mr Knightleylaughing and I agree with you entirely that it will be a muchbetter thing  Invite him to dinner Emma and help him to the bestof the fish and the chicken but leave him to chuse his own wife Depend upon it a man of six or seven and twenty can take careof himself

然后，You can convert it to one token per line with our tokenizer (included in the box) with the following command:

java -cp stanford-ner.jar edu.stanford.nlp.process.PTBTokenizer jane-austen-emma-ch1.txt > jane-austen-emma-ch1.tok

运行完成此命令之后，你可以看到生成了一个jane-austen-emma-ch1.tok文件，此文件的每一行均只有一个单词。
然后需要对训练数据进行实体标记，有多种工具可以使用。可以使用文本文档工具进行手工标记。由于内容太多，可以使用程序对每一行加上制表符（tab）和0，perl的代码如下；然后，再使用文本文档编辑器，对特殊的行进行手工修改标记。
运行下面命令产生jane-austen-emma-ch1.tsv文件（每一行均加上’\t0’字符串）

perl -ne 'chomp; print "$_\tO\n"' jane-austen-emma-ch1.tok > jane-austen-emma-ch1.tsv

如果你不想做上述步骤，可以从这个链接下载tsv文件http://nlp.stanford.edu/software/ner-example/jane-austen-emma-ch1.tsv
我们仅仅标注了一个实体类型——用“PERS”表示人名；但是，你可以在这个数据中加入第二种实体类型，如地理位置“LOC”。你应该确定每行仅仅只包括内容字段和制表符（tab），空格也不能有，额外的制表符（tab）也不能有，否则训练文件解析器可能会出错。一个空行分隔两个“document”，一个“document”只能是一个句子，或者更大一点儿的单元，如段落；这是CRF推断的单位。“document”不能太大，否则你将浪费许多内存，并且还可能碰到很多问题。你仍然可以使用一些测试数据去判断系统是否可以良好运行，这里有jane-austen-emma-ch2.txt（http://nlp.stanford.edu/software/ner-example/jane-austen-emma-ch2.txt）和jane-austen-emma-ch2.tsv（http://nlp.stanford.edu/software/ner-example/jane-austen-emma-ch2.tsv）

新建austen.prop属性文件

#location of the training filetrainFile = jane-austen-emma-ch1.tsv#location where you would like to save (serialize to) your#classifier; adding .gz at the end automatically gzips the file,#making it faster and smallerserializeTo = ner-model.ser.gz#structure of your training file; this tells the classifier#that the word is in column 0 and the correct answer is in#column 1map = word=0,answer=1#these are the features we'd like to train with#some are discussed below, the rest can be#understood by looking at NERFeatureFactoryuseClassFeature=trueuseWord=trueuseNGrams=true#no ngrams will be included that do not contain either the#beginning or end of the wordnoMidNGrams=trueuseDisjunctive=truemaxNGramLeng=6usePrev=trueuseNext=trueuseSequences=trueusePrevSequences=truemaxLeft=1#the next 4 deal with word shape featuresuseTypeSeqs=trueuseTypeSeqs2=trueuseTypeySequences=truewordShape=chris2useLC

然后用下面的命令训练一个分类器

java -cp stanford-ner.jar edu.stanford.nlp.ie.crf.CRFClassifier -prop austen.prop

一旦完成上述命令。将产生一个NER模型文件ner-model.ser.gz，判断它是否有效，使用下面命令来测试

java -cp stanford-ner.jar edu.stanford.nlp.ie.crf.CRFClassifier -loadClassifier ner-model.ser.gz -testFile jane-austen-emma-ch2.tsv

最后得到的结果如下

可以看到第一列是作为输入项的单词列，第二列是正确标注的结果，第三类是由Stanford NER猜的结果。你能够发现此分类器发现了大多数的人名实体，但不是全部的。主要是因为训练数据比较少，而且这只是一个相对基本的特征集。代码最后评估出实体精确分类的性能，用recall（召回率）表示，用符号F1代表。最后的结果F1是85.95%.（不知道为什么我的是77.11%）


所以训练自己的NER模型步骤是：
1)训练数据的源数据
2)指定了你想用的属性的属性文件（这个是可选的，但是有最好）
3)测试文件
对于训练数据的源数据，需要保证每一个单词占据一行，并且附有正确答案（单词与答案之间使用制表符分隔开）。如果要明确指定单词的更多特征，可以将这些特征添加到文件的新列中，然后在属性文件的map 行中设置适当结构。例如，如果你的数据中添加了第三列表示新的特征，则可以写成“map = word = 0，answer = 1，mySpecialFeature = 2”。

Right now, most arbitrarily named features (like mySpecialFeature) will not work without making modifications to the source code. To see which features can already be attached to a CoreLabel, look at edu.stanford.nlp.ling.AnnotationLookup. There is a table which creates a mapping between key and annotation type. For example, if you search in this file for LEMMA_KEY, you will see that lemma produces a LemmaAnnotation. If you have added a new annotation, you can add its type to this table, or you can use one of the known names that already work, like tag, lemma, chunk, web.

If you modify AnnotationLookup, you need to read the data from the column, translate it to the desired object type, and attach it to the CoreLabel using a CoreAnnotation. Quite a few CoreAnnotations are provided in the class appropriately called CoreAnnotations. If the particular one you are looking for is not present, you can add a new subclass by using one of the existing CoreAnnotations as an example.

If the feature you attached to the CoreLabel is not already used as a feature in NERFeatureFactory, you will need to add code that extracts the feature from the CoreLabel and adds it to the feature set. Bear in mind that features must have unique names, or they will conflict with existing features, which is why we add markers such as “-GENIA”, “-PGENIA”, and “-NGENIA” to our features. As long as you choose a unique marker, the feature itself can be any string followed by its marker and will not conflict with any existing features. Processing is done using a bag of features model, with all of the features mixed together, which is why it is important to not have any name conflicts.

Once you’ve annotated your data, you make a properties file with the features you want. You can use the example properties file, and refer to the NERFeatureFactory for more possible features. Finally, you can test on your annotated test data as shown above or annotate more text using the -textFile command rather than -testFile.

备用：在上文中提到了使用perl语言，对文本文档的每一行后面添加一个tab键与0，不仅仅可以使用编程语言perl，python等来完成这样的步骤，其实还有一个更简单的方法，使用sublime text，或者notepad++等这样的文本编辑软件，也可以达到目的。
    给每行首部位置加上内容，用sublime text打开文本文档文件之后，菜单find—replace，打开替换功能，先勾选替换框下部的模式为正则表达式(Regular Expression)，在“查找目标”(Find what)处输入^，在“替换为”(Replace with)处输入想要添加的内容，最后点击按钮“replace all”即可。
    给每行尾部位置加上内容，一样的步骤，在“查找目标”(Find what)处输入$，后面一样。通过这种文本编辑软件来进行处理，速度更快！