后台运行python程序 遇到缓冲区问题

From: http://www.iteye.com/topic/867446

环境：linux

 

一段执行时间很长的程序（用python做hive客户端执行mapreduce） 在linux后台执行，把结果输出到某文件：

 

 

Python代码  

1. python xxx.py > log.log&   

 

 遇到的问题，程序没报错，文件里却什么都没有，没有输入进去。为什么呢？

于是我首先尝试用：

 

 

Java代码  

1. nohup python xxx.py > log.log &  

 

预料之中 还是不行。

 

于是我做实验：

 

写了个test.py:

 

Java代码  

1. import sys,time  
2. from threading import Thread  
3. class worker(Thread):  
4.      def run(self):  
5.          for x in xrange(0,111):  
6.              print x  
7.              time.sleep(1)  
8. def run():  
9.      worker().start()  
10. if __name__ == '__main__':  
11.     run()  

 每秒打印一次

我直接用python text.py 执行  没问题  每秒输出一个数

但我在后台执行：

 

Python代码  

1. python test.py > log.log&   

 

还是不行。开始不输出  直到程序执行完，一次性的写到log.log文件了。

为什么呢？ 

原因可能是python 的print 先写到缓冲区了，还没flush到文件。

于是加了一行“ sys.stdout.flush()” --在每次print后都flush一次。：

 

Java代码  

1. import sys,time  
2. from threading import Thread  
3. class worker(Thread):  
4.      def run(self):  
5.          for x in xrange(0,111):  
6.              print x  
7.              sys.stdout.flush()    
8.              time.sleep(1)  
9. def run():  
10.      worker().start()  
11. if __name__ == '__main__':  
12.     run()  

 

问题解决。

===============================================================================

还可以：python-u xxx.py > log.log & ，再细看下帮助文档：man python

PYTHON(1)                                                            PYTHON(1)


NAME
       python - an interpreted, interactive, object-oriented programming language


SYNOPSIS
       python [ -d ] [ -E ] [ -h ] [ -i ] [ -m module-name ] [ -O ]
              [ -Q argument ] [ -S ] [ -t ] [ -u ]
              [ -v ] [ -V ] [ -W argument ] [ -x ] [ -3 ]
              [ -c command | script | - ] [ arguments ]


DESCRIPTION
       Python  is  an  interpreted,  interactive, object-oriented programming language that combines remarkable power with very clear syntax.
       For an introduction to programming in Python you are referred to the Python Tutorial.  The Python Library Reference documents built-in
       and  standard types, constants, functions and modules.  Finally, the Python Reference Manual describes the syntax and semantics of the
       core language in (perhaps too) much detail.  (These documents may be located via the INTERNET RESOURCES below; they may  be  installed
       on your system as well.)


       Python’s  basic  power  can  be  extended  with your own modules written in C or C++.  On most systems such modules may be dynamically
       loaded.  Python is also adaptable as an extension language for existing applications.  See the internal documentation for hints.


       Documentation for installed Python modules and packages can be viewed by running the pydoc program.

COMMAND LINE OPTIONS
       -c command
              Specify the command to execute (see next section).  This terminates the option list (following options are passed as  arguments
              to the command).


       -d     Turn on parser debugging output (for wizards only, depending on compilation options).


       -E     Ignore environment variables like PYTHONPATH and PYTHONHOME that modify the behavior of the interpreter.


       -h     Prints the usage for the interpreter executable and exits.


       -i     When  a  script  is passed as first argument or the -c option is used, enter interactive mode after executing the script or the
              command.  It does not read the $PYTHONSTARTUP file.  This can be useful to inspect global variables or a  stack  trace  when  a
              script raises an exception.


       -m module-name
              Searches sys.path for the named module and runs the corresponding .py file as a script.

      -u    Force stdin, stdout and stderr to be totallyunbuffered.  On systems where it matters, also put stdin,  stdout  and  stderr  in

              binary  mode.   Note  that  there  is  internal  buffering in xreadlines(), readlines() and file-object iterators ("for line in
              sys.stdin") which is not influenced by this option.  To work around this, you will want to use "sys.stdin.readline()" inside  a
              "while 1:" loop.


       -v     Print  a  message  each  time a module is initialized, showing the place (filename or built-in module) from which it is loaded.
              When given twice, print a message for each file that is checked for when searching for a module.  Also provides information  on
              module cleanup at exit.


       -V     Prints the Python version number of the executable and exits.