python的Tqdm模块

来源：互联网发布：紫色水离子淘宝店地址编辑：程序博客网时间：2024/05/22 08:07

Tqdm 是一个快速，可扩展的Python进度条，可以在 Python 长循环中添加一个进度提示信息，用户只需要封装任意的迭代器 tqdm(iterator)。

我的系统是window环境，首先安装python，接下来就是pip。

pip安装：

在python根目录下创建一个get-pip.py的文件，内容：

https://bootstrap.pypa.io/get-pip.py

然后在CMD窗口进入python下面：

输出：

python -m pip install -U pip

由于Tqdm要求的pip版本是9.0所以需要手动安装pip9.0

http://pypi.python.org/pypi/pip

下载安装包9.0

然后解压进入，CMD窗口输入：python setup.py install

然后就可以安装Tqdm了，

pip install tqdm

安装最新的开发版的话

pip install -e git+https://github.com/tqdm/tqdm.git@master#egg=tqdm

最后看看怎么用呢？https://pypi.python.org/pypi/tqdm

基本用法：

from tqdm import tqdmfor i in tqdm(range(10000)):     sleep(0.01)

当然除了tqdm，还有trange,使用方式完全相同

for i in trange(100):        sleep(0.1)

只要传入list都可以：

pbar = tqdm(["a", "b", "c", "d"])for char in pbar:    pbar.set_description("Processing %s" % char)

也可以手动控制更新

with tqdm(total=100) as pbar:    for i in range(10):        pbar.update(10)

也可以这样：

pbar = tqdm(total=100)for i in range(10):    pbar.update(10)pbar.close()

在Shell的tqdm用法

统计所有python脚本的行数：

$ time find . -name '*.py' -exec cat \{} \; | wc -l857365real    0m3.458suser    0m0.274ssys     0m3.325s$ time find . -name '*.py' -exec cat \{} \; | tqdm | wc -l857366it [00:03, 246471.31it/s]857365real    0m3.585suser    0m0.862ssys     0m3.358s

使用参数：

$ find . -name '*.py' -exec cat \{} \; |    tqdm --unit loc --unit_scale --total 857366 >> /dev/null100%|███████████████████████████████████| 857K/857K [00:04<00:00, 246Kloc/s]

备份一个目录：

$ 7z a -bd -r backup.7z docs/ | grep Compressing |    tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log100%|███████████████████████████████▉| 8014/8014 [01:37<00:00, 82.29files/s]

通过看示范的代码，我们能发现使用的核心是tqdm和trange这两个函数，从代码层面分析tqdm的功能，那首先是init.py

__all__ = ['tqdm', 'tqdm_gui', 'trange', 'tgrange', 'tqdm_pandas',           'tqdm_notebook', 'tnrange', 'main', 'TqdmKeyError', 'TqdmTypeError',           '__version__']

跟踪到_tqdm.py，能看到tqdm类的声明，首先是初始化

def __init__(self, iterable=None, desc=None, total=None, leave=True,                 file=sys.stderr, ncols=None, mininterval=0.1,                 maxinterval=10.0, miniters=None, ascii=None, disable=False,                 unit='it', unit_scale=False, dynamic_ncols=False,                 smoothing=0.3, bar_format=None, initial=0, position=None,                 gui=False, **kwargs):

Parametersiterable : iterable, optionalIterable to decorate with a progressbar.可迭代的进度条。Leave blank to manually manage the updates.留空手动管理更新？？desc : str, optionalPrefix for the progressbar.进度条的描述total : int, optionalThe number of expected iterations. If unspecified,len(iterable) is used if possible. As a last resort, only basicprogress statistics are displayed (no ETA, no progressbar).If gui is True and this parameter needs subsequent updating,specify an initial arbitrary large positive integer,e.g. int(9e9).预期的迭代数目，默认为None，则尽可能的迭代下去，如果gui设置为True，这里则需要后续的更新，将需要指定为一个初始随意值较大的正整数，例如int(9e9)leave : bool, optionalIf [default: True], keeps all traces of the progressbarupon termination of iteration.保留进度条存在的痕迹，简单来说就是会把进度条的最终形态保留下来，默认为Truefile : io.TextIOWrapper or io.StringIO, optionalSpecifies where to output the progress messages[default: sys.stderr]. Uses file.write(str) and file.flush()methods.指定消息的输出ncols : int, optionalThe width of the entire output message. If specified,dynamically resizes the progressbar to stay within this bound.If unspecified, attempts to use environment width. Thefallback is a meter width of 10 and no limit for the counter andstatistics. If 0, will not print any meter (only stats).整个输出消息的宽度。如果指定，动态调整的进度停留在这个边界。如果未指定，尝试使用环境的宽度。如果为0，将不打印任何东西（只统计）。mininterval : float, optionalMinimum progress update interval, in seconds [default: 0.1].最小进度更新间隔，以秒为单位（默认值：0.1）。maxinterval : float, optionalMaximum progress update interval, in seconds [default: 10.0].最大进度更新间隔，以秒为单位（默认值：10）。miniters : int, optionalMinimum progress update interval, in iterations.If specified, will set mininterval to 0.最小进度更新周期ascii : bool, optionalIf unspecified or False, use unicode (smooth blocks) to fillthe meter. The fallback is to use ASCII characters 1-9 #.如果不设置，默认为unicode编码disable : bool, optionalWhether to disable the entire progressbar wrapper[default: False].是否禁用整个进度条包装(如果为True，进度条不显示)unit : str, optionalString that will be used to define the unit of each iteration[default: it].将被用来定义每个单元的字符串？？？unit_scale : bool, optionalIf set, the number of iterations will be reduced/scaledautomatically and a metric prefix following theInternational System of Units standard will be added(kilo, mega, etc.) [default: False].如果设置，迭代的次数会自动按照十、百、千来添加前缀，默认为falsedynamic_ncols : bool, optionalIf set, constantly alters ncols to the environment (allowingfor window resizes) [default: False].不断改变ncols环境，允许调整窗口大小smoothing : float, optionalExponential moving average smoothing factor for speed estimates(ignored in GUI mode). Ranges from 0 (average speed) to 1(current/instantaneous speed) [default: 0.3].bar_format : str, optionalSpecify a custom bar string formatting. May impact performance.If unspecified, will use ‘{l_bar}{bar}{r_bar}’, where l_bar is‘{desc}{percentage:3.0f}%|’ and r_bar is‘| {n_fmt}/{total_fmt} [{elapsed_str}<{remaining_str}, {rate_fmt}]’Possible vars: bar, n, n_fmt, total, total_fmt, percentage,rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc.自定义栏字符串格式化…默认会使用{l_bar}{bar}{r_bar}的格式，格式同上initial : int, optionalThe initial counter value. Useful when restarting a progressbar [default: 0].初始计数器值，默认为0position : int, optionalSpecify the line offset to print this bar (starting from 0)Automatic if unspecified.Useful to manage multiple bars at once (eg, from threads).指定偏移，这个功能在多个条中有用gui : bool, optionalWARNING: internal parameter - do not use.Use tqdm_gui(…) instead. If set, will attempt to usematplotlib animations for a graphical output [default: False].内部参数…Returnsout : decorated iterator.返回为一个迭代器

其实不用分析更多代码，多看看几个例子：(官网的例子)

7zx.py压缩进度条

# -*- coding: utf-8 -*-"""Usage:  7zx.py [--help | options] <zipfiles>...Options:  -h, --help     Print this help and exit  -v, --version  Print version and exit  -c, --compressed       Use compressed (instead of uncompressed) file sizes  -s, --silent   Do not print one row per zip file  -y, --yes      Assume yes to all queries (for extraction)  -D=<level>, --debug=<level>                 Print various types of debugging information. Choices:                         CRITICAL|FATAL                         ERROR                         WARN(ING)                         [default: INFO]                         DEBUG                         NOTSET  -d, --debug-trace      Print lots of debugging information (-D NOTSET)"""from __future__ import print_functionfrom docopt import docoptimport logging as logimport subprocessimport refrom tqdm import tqdmimport ptyimport osimport io__author__ = "Casper da Costa-Luis <casper.dcl@physics.org>"__licence__ = "MPLv2.0"__version__ = "0.2.0"__license__ = __licence__RE_SCN = re.compile("([0-9]+)\s+([0-9]+)\s+(.*)$", flags=re.M)def main():    args = docopt(__doc__, version=__version__)    if args.pop('--debug-trace', False):        args['--debug'] = "NOTSET"    log.basicConfig(level=getattr(log, args['--debug'], log.INFO),                    format='%(levelname)s: %(message)s')    log.debug(args)    # Get compressed sizes    zips = {}    for fn in args['<zipfiles>']:        info = subprocess.check_output(["7z", "l", fn]).strip()        finfo = RE_SCN.findall(info)        # builtin test: last line should be total sizes        log.debug(finfo)        totals = map(int, finfo[-1][:2])        # log.debug(totals)        for s in range(2):            assert(sum(map(int, (inf[s] for inf in finfo[:-1]))) == totals[s])        fcomp = dict((n, int(c if args['--compressed'] else u))                     for (u, c, n) in finfo[:-1])        # log.debug(fcomp)        # zips  : {'zipname' : {'filename' : int(size)}}        zips[fn] = fcomp    # Extract    cmd7zx = ["7z", "x", "-bd"]    if args['--yes']:        cmd7zx += ["-y"]    log.info("Extracting from {:d} file(s)".format(len(zips)))    with tqdm(total=sum(sum(fcomp.values()) for fcomp in zips.values()),              unit="B", unit_scale=True) as tall:        for fn, fcomp in zips.items():            md, sd = pty.openpty()            ex = subprocess.Popen(cmd7zx + [fn],                                  bufsize=1,                                  stdout=md,  # subprocess.PIPE,                                  stderr=subprocess.STDOUT)            os.close(sd)            with io.open(md, mode="rU", buffering=1) as m:                with tqdm(total=sum(fcomp.values()), disable=len(zips) < 2,                          leave=False, unit="B", unit_scale=True) as t:                    while True:                        try:                            l_raw = m.readline()                        except IOError:                            break                        l = l_raw.strip()                        if l.startswith("Extracting"):                            exname = l.lstrip("Extracting").lstrip()                            s = fcomp.get(exname, 0)  # 0 is likely folders                            t.update(s)                            tall.update(s)                        elif l:                            if not any(l.startswith(i) for i in                                       ("7-Zip ",                                        "p7zip Version ",                                        "Everything is Ok",                                        "Folders: ",                                        "Files: ",                                        "Size: ",                                        "Compressed: ")):                                if l.startswith("Processing archive: "):                                    if not args['--silent']:                                        t.write(t.format_interval(                                            t.start_t - tall.start_t) + ' ' +                                            l.lstrip("Processing archive: "))                                else:                                    t.write(l)            ex.wait()main.__doc__ = __doc__if __name__ == "__main__":    main()

tqdm_wget.py

"""An example of wrapping manual tqdm updates for urllib reporthook.# urllib.urlretrieve documentation> If present, the hook function will be called once> on establishment of the network connection and once after each block read> thereafter. The hook will be passed three arguments; a count of blocks> transferred so far, a block size in bytes, and the total size of the file.Usage:    tqdm_wget.py [options]Options:-h, --help    Print this help message and exit-u URL, --url URL  : string, optional    The url to fetch.    [default: http://www.doc.ic.ac.uk/~cod11/matryoshka.zip]-o FILE, --output FILE  : string, optional    The local file path in which to save the url [default: /dev/null]."""import urllibfrom tqdm import tqdmfrom docopt import docoptdef my_hook(t):    """    Wraps tqdm instance. Don't forget to close() or __exit__()    the tqdm instance once you're done with it (easiest using `with` syntax).    Example    -------    >>> with tqdm(...) as t:    ...     reporthook = my_hook(t)    ...     urllib.urlretrieve(..., reporthook=reporthook)    """    last_b = [0]    def inner(b=1, bsize=1, tsize=None):        """        b  : int, optional            Number of blocks just transferred [default: 1].        bsize  : int, optional            Size of each block (in tqdm units) [default: 1].        tsize  : int, optional            Total size (in tqdm units). If [default: None] remains unchanged.        """        if tsize is not None:            t.total = tsize        t.update((b - last_b[0]) * bsize)        last_b[0] = b    return inneropts = docopt(__doc__)eg_link = opts['--url']eg_file = eg_link.replace('/', ' ').split()[-1]with tqdm(unit='B', unit_scale=True, leave=True, miniters=1,          desc=eg_file) as t:  # all optional kwargs    urllib.urlretrieve(eg_link, filename=opts['--output'],                       reporthook=my_hook(t), data=None)

examples.py

"""# Simple tqdm examples and profiling# Benchmarkfor i in _range(int(1e8)):    pass# Basic demoimport tqdmfor i in tqdm.trange(int(1e8)):    pass# Some decorationsimport tqdmfor i in tqdm.trange(int(1e8), miniters=int(1e6), ascii=True,                     desc="cool", dynamic_ncols=True):    pass# Nested barsfrom tqdm import trangefor i in trange(10):    for j in trange(int(1e7), leave=False, unit_scale=True):        pass# Experimental GUI demoimport tqdmfor i in tqdm.tgrange(int(1e8)):    pass# Comparison to https://code.google.com/p/python-progressbar/try:    from progressbar.progressbar import ProgressBarexcept ImportError:    passelse:    for i in ProgressBar()(_range(int(1e8))):        pass# Dynamic miniters benchmarkfrom tqdm import trangefor i in trange(int(1e8), miniters=None, mininterval=0.1, smoothing=0):    pass# Fixed miniters benchmarkfrom tqdm import trangefor i in trange(int(1e8), miniters=4500000, mininterval=0.1, smoothing=0):    pass"""from time import sleepfrom timeit import timeitimport re# Simple demofrom tqdm import trangefor i in trange(16, leave=True):    sleep(0.1)# Profiling/overhead testsstmts = filter(None, re.split(r'\n\s*#.*?\n', __doc__))for s in stmts:    print(s.replace('import tqdm\n', ''))    print(timeit(stmt='try:\n\t_range = xrange'                      '\nexcept:\n\t_range = range\n' + s, number=1),          'seconds')

pandas_progress_apply.py

import pandas as pdimport numpy as npfrom tqdm import tqdmdf = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`# (can use `tqdm_gui`, `tqdm_notebook`, optional kwargs, etc.)tqdm.pandas(desc="my bar!")# Now you can use `progress_apply` instead of `apply`# and `progress_map` instead of `map`df.progress_apply(lambda x: x**2)# can also groupby:# df.groupby(0).progress_apply(lambda x: x**2)# -- Source code for `tqdm_pandas` (really simple!)# def tqdm_pandas(t):#   from pandas.core.frame import DataFrame#   def inner(df, func, *args, **kwargs):#       t.total = groups.size // len(groups)#       def wrapper(*args, **kwargs):#           t.update(1)#           return func(*args, **kwargs)#       result = df.apply(wrapper, *args, **kwargs)#       t.close()#       return result#   DataFrame.progress_apply = inner

引用tqdm并非强制作为依赖：

include_no_requirements.py

# How to import tqdm without enforcing it as a dependencytry:    from tqdm import tqdmexcept ImportError:    def tqdm(*args, **kwargs):        if args:            return args[0]        return kwargs.get('iterable', None)

参考：
https://github.com/tqdm/tqdm/tree/master/examples

https://pypi.python.org/pypi/tqdm

https://github.com/tqdm/tqdm

0 0