另外,我们希望考虑到时间序列和横截面数据集的典型方向的通用API函数的合理默认行为。当使用ndarrays存储二维和三维数据时,用户在编写函数时要考虑数据集的方向,轴被认为或多或少是等价的(除非C或Fortran连续性对于性能而言很重要)。在熊猫中,这些轴旨在为数据提供更多的语义含义; 即对于一个特定的数据集,可能有一个“正确”的方式来定位数据。那么目标就是减少编码下游函数中的数据转换所需的精力。
例如,对于表格数据(DataFrame),在考虑索引(行)和列而不是轴0和轴1时,它在语义上更有用。然后遍历DataFrame的列.。
对象的创建:
传入一系列的值来创建一个series:
In [4]: s = pd.Series([1,3,5,np.nan,6,8])In [5]: sOut[5]: 0 1.01 3.02 5.03 NaN4 6.05 8.0dtype: float64
传入一个numpy数组来创建一个Dataframe
In [6]: dates = pd.date_range('20130101', periods=6)In [7]: datesOut[7]: DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04', '2013-01-05', '2013-01-06'], dtype='datetime64[ns]', freq='D')In [8]: df = pd.DataFrame(np.random.randn(6,4), index=dates, columns=list('ABCD'))In [9]: dfOut[9]: A B C D2013-01-01 0.469112 -0.282863 -1.509059 -1.1356322013-01-02 1.212112 -0.173215 0.119209 -1.0442362013-01-03 -0.861849 -2.104569 -0.494929 1.0718042013-01-04 0.721555 -0.706771 -1.039575 0.2718602013-01-05 -0.424972 0.567020 0.276232 -1.0874012013-01-06 -0.673690 0.113648 -1.478427 0.524988
DataFrame
通过传递一个可以转换为一系列对象的字典来创建
In [10]: df2 = pd.DataFrame({ 'A' : 1., ....: 'B' : pd.Timestamp('20130102'), ....: 'C' : pd.Series(1,index=list(range(4)),dtype='float32'), ....: 'D' : np.array([3] * 4,dtype='int32'), ....: 'E' : pd.Categorical(["test","train","test","train"]), ....: 'F' : 'foo' }) ....: In [11]: df2Out[11]: A B C D E F0 1.0 2013-01-02 1.0 3 test foo1 1.0 2013-01-02 1.0 3 train foo2 1.0 2013-01-02 1.0 3 test foo3 1.0 2013-01-02 1.0 3 train foo
查看框架的顶部和底部的行
In [14]: df.head()Out[14]: A B C D2013-01-01 0.469112 -0.282863 -1.509059 -1.1356322013-01-02 1.212112 -0.173215 0.119209 -1.0442362013-01-03 -0.861849 -2.104569 -0.494929 1.0718042013-01-04 0.721555 -0.706771 -1.039575 0.2718602013-01-05 -0.424972 0.567020 0.276232 -1.087401In [15]: df.tail(3)Out[15]: A B C D2013-01-04 0.721555 -0.706771 -1.039575 0.2718602013-01-05 -0.424972 0.567020 0.276232 -1.0874012013-01-06 -0.673690 0.113648 -1.478427 0.524988
显示索引,列和底层numpy数据
In [16]: df.indexOut[16]: DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04', '2013-01-05', '2013-01-06'], dtype='datetime64[ns]', freq='D')In [17]: df.columnsOut[17]: Index(['A', 'B', 'C', 'D'], dtype='object')In [18]: df.valuesOut[18]: array([[ 0.4691, -0.2829, -1.5091, -1.1356], [ 1.2121, -0.1732, 0.1192, -1.0442], [-0.8618, -2.1046, -0.4949, 1.0718], [ 0.7216, -0.7068, -1.0396, 0.2719], [-0.425 , 0.567 , 0.2762, -1.0874], [-0.6737, 0.1136, -1.4784, 0.525 ]])
转置数据
In [20]: df.TOut[20]: 2013-01-01 2013-01-02 2013-01-03 2013-01-04 2013-01-05 2013-01-06A 0.469112 1.212112 -0.861849 0.721555 -0.424972 -0.673690B -0.282863 -0.173215 -2.104569 -0.706771 0.567020 0.113648C -1.509059 0.119209 -0.494929 -1.039575 0.276232 -1.478427D -1.135632 -1.044236 1.071804 0.271860 -1.087401 0.524988
按轴排序
In [21]: df.sort_index(axis=1, ascending=False)Out[21]: D C B A2013-01-01 -1.135632 -1.509059 -0.282863 0.4691122013-01-02 -1.044236 0.119209 -0.173215 1.2121122013-01-03 1.071804 -0.494929 -2.104569 -0.8618492013-01-04 0.271860 -1.039575 -0.706771 0.7215552013-01-05 -1.087401 0.276232 0.567020 -0.4249722013-01-06 0.524988 -1.478427 0.113648 -0.673690
按值排序
In [22]: df.sort_values(by='B')Out[22]: A B C D2013-01-03 -0.861849 -2.104569 -0.494929 1.0718042013-01-04 0.721555 -0.706771 -1.039575 0.2718602013-01-01 0.469112 -0.282863 -1.509059 -1.1356322013-01-02 1.212112 -0.173215 0.119209 -1.0442362013-01-06 -0.673690 0.113648 -1.478427 0.5249882013-01-05 -0.424972 0.567020 0.276232 -1.087401
通过[]
,切片行
In [24]: df[0:3]Out[24]: A B C D2013-01-01 0.469112 -0.282863 -1.509059 -1.1356322013-01-02 1.212112 -0.173215 0.119209 -1.0442362013-01-03 -0.861849 -2.104569 -0.494929 1.071804In [25]: df['20130102':'20130104']Out[25]: A B C D2013-01-02 1.212112 -0.173215 0.119209 -1.0442362013-01-03 -0.861849 -2.104569 -0.494929 1.0718042013-01-04 0.721555 -0.706771 -1.039575 0.271860
使用标签获取横截面
In [26]: df.loc[dates[0]]Out[26]: A 0.469112B -0.282863C -1.509059D -1.135632Name: 2013-01-01 00:00:00, dtype: float64
可以包括分类数据在一个DataFram中
df = pd.DataFrame({"id":[1,2,3,4,5,6], "raw_grade":['a', 'b', 'b', 'a', 'a', 'e']})
df["grade"] = df["raw_grade"].astype("category")df["grade"]Out: 0 a1 b2 b3 a4 a5 eName: grade, dtype: categoryCategories (3, object): [a, b, e]
写入到csv中
df.to_csv('foo.csv')
排序是按类别排序的
df.sort_values(by="grade") id raw_grade grade5 6 e very bad1 2 b good2 3 b good0 1 a very good3 4 a very good4 5 a very good
可以在频率转换过程中执行重采样操作
rng = pd.date_range('1/1/2012', periods=100, freq='S')ts = pd.Series(np.random.randint(0, 500, len(rng)), index=rng)ts.resample('5Min').sum()Out: 2012-01-01 25083Freq: 5T, dtype: int64
按多列分组会形成一个分层索引
df.groupby(['A','B']).sum()Out: C DA B bar one -1.814470 2.395985 three -0.595447 0.166599 two -0.392670 -0.136473foo one -1.195665 -0.616981 three 1.928123 -1.623033 two 2.414034 1.600434
分组,然后将函数sum
应用于结果
df.groupby('A').sum() C DA bar -2.802588 2.42611foo 3.146492 -0.63958
通过“group by”,我们指的是涉及一个或多个以下步骤的过程
- 拆分数据到基于某些标准组
- 独立地为每个组应用一个功能
- 将结果组合成一个数据结构
将行附加到dataframe
In [87]: df = pd.DataFrame(np.random.randn(8, 4), columns=['A','B','C','D'])In [88]: dfOut[88]: A B C D0 1.346061 1.511763 1.627081 -0.9905821 -0.441652 1.211526 0.268520 0.0245802 -1.577585 0.396823 -0.105381 -0.5325323 1.453749 1.208843 -0.080952 -0.2646104 -0.727965 -0.589346 0.339969 -0.6932055 -0.339355 0.593616 0.884345 1.5914316 0.141809 0.220390 0.435589 0.1924517 -0.096701 0.803351 1.715071 -0.708758In [89]: s = df.iloc[3]In [90]: df.append(s, ignore_index=True)Out[90]: A B C D0 1.346061 1.511763 1.627081 -0.9905821 -0.441652 1.211526 0.268520 0.0245802 -1.577585 0.396823 -0.105381 -0.5325323 1.453749 1.208843 -0.080952 -0.2646104 -0.727965 -0.589346 0.339969 -0.6932055 -0.339355 0.593616 0.884345 1.5914316 0.141809 0.220390 0.435589 0.1924517 -0.096701 0.803351 1.715071 -0.7087588 1.453749 1.208843 -0.080952 -0.264610
在连接/合并类型操作的情况下,熊猫提供了各种功能,可以方便地将Series,DataFrame和Panel对象与索引和关系代数功能的各种设置逻辑组合在一起
In [73]: df = pd.DataFrame(np.random.randn(10, 4))In [74]: dfOut[74]: 0 1 2 30 -0.548702 1.467327 -1.015962 -0.4830751 1.637550 -1.217659 -0.291519 -1.7455052 -0.263952 0.991460 -0.919069 0.2660463 -0.709661 1.669052 1.037882 -1.7057754 -0.919854 -0.042379 1.247642 -0.0099205 0.290213 0.495767 0.362949 1.5481066 -1.131345 -0.089329 0.337863 -0.9458677 -0.932132 1.956030 0.017587 -0.0166928 -0.575247 0.254161 -1.143704 0.2158979 1.193555 -0.077118 -0.408530 -0.862495# break it into piecesIn [75]: pieces = [df[:3], df[3:7], df[7:]]In [76]: pd.concat(pieces)Out[76]: 0 1 2 30 -0.548702 1.467327 -1.015962 -0.4830751 1.637550 -1.217659 -0.291519 -1.7455052 -0.263952 0.991460 -0.919069 0.2660463 -0.709661 1.669052 1.037882 -1.7057754 -0.919854 -0.042379 1.247642 -0.0099205 0.290213 0.495767 0.362949 1.5481066 -1.131345 -0.089329 0.337863 -0.9458677 -0.932132 1.956030 0.017587 -0.0166928 -0.575247 0.254161 -1.143704 0.2158979 1.193555 -0.077118 -0.408530 -0.862495
删除任何缺少数据的行
df1.dropna(how='any')Out: A B C D F E2013-01-02 1.212112 -0.173215 0.119209 5 1.0 1.0
填写缺少的数据
df1.fillna(value=5)Out: A B C D F E2013-01-01 0.000000 0.000000 -1.509059 5 5.0 1.02013-01-02 1.212112 -0.173215 0.119209 5 1.0 1.02013-01-03 -0.861849 -2.104569 -0.494929 5 2.0 5.02013-01-04 0.721555 -0.706771 -1.039575 5 3.0 5.0