pandas官方文档cookbook（6）中Split&Pivot&Apply翻译

来源：互联网发布：python java socket 编辑：程序博客网时间：2024/06/03 20:28

文档版本：0.20.3
这些例子是用python3.4写出来的。对于较早的python版本需要对代码做些相应的调整。
Pandas（pd）和Numpy（np）是唯一两个默认导入的包。其余的包会显示导入给新用户看。
若有翻译不当的地方，请多多指教。

这份文档中的例子都是从Stack-Overflow和Github中别人提问的比较经典的问题，作者从中进行提炼与总结。

分割

通过对行的逻辑描绘分割数据框，使其成为一个数据框的列表。

In [125]: df = pd.DataFrame(data={'Case' : ['A','A','A','B','A','A','B','A','A'],   .....:                         'Data' : np.random.randn(9)})   .....: In [126]: dfs = list(zip(*df.groupby((1*(df['Case']=='B')).cumsum().rolling(window=3,min_periods=1).median())))[-1]In [127]: dfs[0]Out[127]:     Case      Data0    A  0.1740681    A -0.4394612    A -0.7413433    B -0.079673In [128]: dfs[1]Out[128]:   Case      Data4    A -0.9228755    A  0.3036386    B -0.917368In [129]: dfs[2]Out[129]:   Case      Data7    A -1.6240628    A -0.758514

枢纽表

部分的加总和小记

In [130]: df = pd.DataFrame(data={'Province' : ['ON','QC','BC','AL','AL','MN','ON'],   .....:                          'City' : ['Toronto','Montreal','Vancouver','Calgary','Edmonton','Winnipeg','Windsor'],   .....:                          'Sales' : [13,6,16,8,4,3,1]})   .....: In [131]: table = pd.pivot_table(df,values=['Sales'],index=['Province'],columns=['City'],aggfunc=np.sum,margins=True)In [132]: table.stack('City')Out[132]:                     SalesProvince City            AL       All         12.0         Calgary      8.0         Edmonton     4.0BC       All         16.0         Vancouver   16.0MN       All          3.0         Winnipeg     3.0...                   ...All      Calgary      8.0         Edmonton     4.0         Montreal     6.0         Toronto     13.0         Vancouver   16.0         Windsor      1.0         Winnipeg     3.0[20 rows x 1 columns]

像R中的plyr的频率表

In [133]: grades = [48,99,75,80,42,80,72,68,36,78]In [134]: df = pd.DataFrame( {'ID': ["x%d" % r for r in range(10)],   .....:                     'Gender' : ['F', 'M', 'F', 'M', 'F', 'M', 'F', 'M', 'M', 'M'],   .....:                     'ExamYear': ['2007','2007','2007','2008','2008','2008','2008','2009','2009','2009'],   .....:                     'Class': ['algebra', 'stats', 'bio', 'algebra', 'algebra', 'stats', 'stats', 'algebra', 'bio', 'bio'],   .....:                     'Participated': ['yes','yes','yes','yes','no','yes','yes','yes','yes','yes'],   .....:                     'Passed': ['yes' if x > 50 else 'no' for x in grades],   .....:                     'Employed': [True,True,True,False,False,False,False,True,True,False],   .....:                     'Grade': grades})   .....: In [135]: df.groupby('ExamYear').agg({'Participated': lambda x: x.value_counts()['yes'],   .....:                     'Passed': lambda x: sum(x == 'yes'),   .....:                     'Employed' : lambda x : sum(x),   .....:                     'Grade' : lambda x : sum(x) / len(x)})   .....: Out[135]:       Participated  Passed  Employed      GradeExamYear                                           2007                 3       2         3  74.0000002008                 3       3         0  68.5000002009                 3       2         2  60.666667

创建一个年和月的交叉列表

In [136]: df = pd.DataFrame({'value': np.random.randn(36)},   .....:                   index=pd.date_range('2011-01-01', freq='M', periods=36))   .....: In [137]: pd.pivot_table(df, index=df.index.month, columns=df.index.year,   .....:                values='value', aggfunc='sum')   .....: Out[137]:         2011      2012      20131  -0.560859  0.120930  0.5168702  -0.589005 -0.210518  0.3431253  -1.070678 -0.931184  2.1378274  -1.681101  0.240647  0.4524295   0.403776 -0.027462  0.4831036   0.609862  0.033113  0.0614957   0.387936 -0.658418  0.2407678   1.815066  0.324102  0.7824139   0.705200 -1.403048  0.62846210 -0.668049 -0.581967 -0.88062711  0.242501 -1.233862  0.77757512  0.313421 -3.520876 -0.779367

Apply

通过对嵌套列表的数据框进行循环处理使其转化为不含列表的多重索引结构

In [138]: df = pd.DataFrame(data={'A' : [[2,4,8,16],[100,200],[10,20,30]], 'B' : [['a','b','c'],['jj','kk'],['ccc']]},index=['I','II','III'])In [139]: def SeriesFromSubList(aList):   .....:    return pd.Series(aList)   .....: In [140]: df_orgz = pd.concat(dict([ (ind,row.apply(SeriesFromSubList)) for ind,row in df.iterrows() ]))

从一个从Series的标量被返回之前对多个列进行循环应用用于计算Series的函数

In [141]: df = pd.DataFrame(data=np.random.randn(2000,2)/10000,   .....:                   index=pd.date_range('2001-01-01',periods=2000),   .....:                   columns=['A','B']); df   .....: Out[141]:                    A         B2001-01-01  0.000032 -0.0000042001-01-02 -0.000001  0.0002072001-01-03  0.000120 -0.0002202001-01-04 -0.000083 -0.0001652001-01-05 -0.000047  0.0001562001-01-06  0.000027  0.0001042001-01-07  0.000041 -0.000101...              ...       ...2006-06-17 -0.000034  0.0000342006-06-18  0.000002  0.0001662006-06-19  0.000023 -0.0000812006-06-20 -0.000061  0.0000122006-06-21 -0.000111  0.0000272006-06-22 -0.000061 -0.0000092006-06-23  0.000074 -0.000138[2000 rows x 2 columns]In [142]: def gm(aDF,Const):   .....:    v = ((((aDF.A+aDF.B)+1).cumprod())-1)*Const   .....:    return (aDF.index[0],v.iloc[-1])   .....: In [143]: S = pd.Series(dict([ gm(df.iloc[i:min(i+51,len(df)-1)],5) for i in range(len(df)-50) ])); SOut[143]: 2001-01-01   -0.0013732001-01-02   -0.0017052001-01-03   -0.0028852001-01-04   -0.0029872001-01-05   -0.0023842001-01-06   -0.0047002001-01-07   -0.005500            ...   2006-04-28   -0.0026822006-04-29   -0.0024362006-04-30   -0.0026022006-05-01   -0.0017852006-05-02   -0.0017992006-05-03   -0.0006052006-05-04   -0.000541Length: 1950, dtype: float64

对多重列进行循环应用返回标量的函数

In [144]: rng = pd.date_range(start = '2014-01-01',periods = 100)In [145]: df = pd.DataFrame({'Open' : np.random.randn(len(rng)),   .....:                    'Close' : np.random.randn(len(rng)),   .....:                    'Volume' : np.random.randint(100,2000,len(rng))}, index=rng); df   .....: Out[145]:                Close      Open  Volume2014-01-01 -0.653039  0.011174    15812014-01-02  1.314205  0.214258    17072014-01-03 -0.341915 -1.046922    17682014-01-04 -1.303586 -0.752902     8362014-01-05  0.396288 -0.410793     6942014-01-06 -0.548006  0.648401     7962014-01-07  0.481380  0.737320     265...              ...       ...     ...2014-04-04 -2.548128  0.120378     5642014-04-05  0.223346  0.231661    19082014-04-06  1.228841  0.952664    10902014-04-07  0.552784 -0.176090    18132014-04-08 -0.795389  1.781318    11032014-04-09 -0.018815 -0.753493    14562014-04-10  1.138197 -1.047997    1193[100 rows x 3 columns]In [146]: def vwap(bars): return ((bars.Close*bars.Volume).sum()/bars.Volume.sum())In [147]: window = 5In [148]: s = pd.concat([ (pd.Series(vwap(df.iloc[i:i+window]), index=[df.index[i+window]])) for i in range(len(df)-window) ]);In [149]: s.round(2)Out[149]: 2014-01-06   -0.032014-01-07    0.072014-01-08   -0.402014-01-09   -0.812014-01-10   -0.632014-01-11   -0.862014-01-12   -0.36          ... 2014-04-04   -1.272014-04-05   -1.362014-04-06   -0.732014-04-07    0.042014-04-08    0.212014-04-09    0.072014-04-10    0.25Length: 95, dtype: float64

pandas的cookbook中翻译到现在还剩下一些作图、读写文件的部分没有翻译，打算弃坑以后有兴趣再来填。

阅读全文

0 0