Powershell: Get Content Faster with ReadCount!
来源:互联网 发布:h耽美网络剧百度云 编辑:程序博客网 时间:2024/05/16 10:49
Do you use Powershell? Do you use Get-Content in Powershell to read files? Do you sometimes work with large text files?
If you answered yes to any of the questions above, then read on - this post is for you!
I have a very simple tip that I used today in a script I was writing. Thought I'd share.
Let's say you have a large text file, such as a packet log from a DNS server that you're debugging. It might be 300 megabytes and millions of lines. I was writing a script to parse the file and collect some statistics that I was after.
When I ran this script against a 52MB text file, the script executed in about 22 seconds. When I ran the script on a 150MB text file, Powershell proceeded to consume over 3GB of RAM within a few seconds, the script never finished, and after bringing my laptop (Win7 x64, 4GB RAM, 4CPU, PS v3, .NET 4.5) to a crawl for about 5 minutes, Powershell just gave up and returned to the prompt without outputting anything. I guess it was some sort of memory leak. But come on... a 150MB file is not even that big...
So I started looking through the help for Get-Content, and it turns out there's an easy workaround:
Now when I run the script against the 52MB file, it completes in 2.8 seconds, and when I run it on the 150MB text file, it finishes in 10.2 seconds!
If you answered yes to any of the questions above, then read on - this post is for you!
I have a very simple tip that I used today in a script I was writing. Thought I'd share.
Let's say you have a large text file, such as a packet log from a DNS server that you're debugging. It might be 300 megabytes and millions of lines. I was writing a script to parse the file and collect some statistics that I was after.
$LogFile = Get-Content $FileNameForEach($_ In $LogFile){ Do-Stuff}
When I ran this script against a 52MB text file, the script executed in about 22 seconds. When I ran the script on a 150MB text file, Powershell proceeded to consume over 3GB of RAM within a few seconds, the script never finished, and after bringing my laptop (Win7 x64, 4GB RAM, 4CPU, PS v3, .NET 4.5) to a crawl for about 5 minutes, Powershell just gave up and returned to the prompt without outputting anything. I guess it was some sort of memory leak. But come on... a 150MB file is not even that big...
So I started looking through the help for Get-Content, and it turns out there's an easy workaround:
$LogFile = Get-Content $FileName -ReadCount 0ForEach($_ In $LogFile){ Do-Stuff}
The -ReadCount parameter specifies how many lines of content are sent through the pipeline at a time.
The default is 1. A value of 0 sends all of the content through at one time.
Now when I run the script against the 52MB file, it completes in 2.8 seconds, and when I run it on the 150MB text file, it finishes in 10.2 seconds!
- Powershell: Get Content Faster with ReadCount!
- SharePoint PowerShell: Get Web ID (GUID) with PowerShell
- powershell get-aduser Property
- Ping Servers with PowerShell
- Get-Content 帮助信息
- get file content
- Android Get Share Content
- PowerShell Get-Eventlog Remote Computer
- Exchange2013 Get-MessageTrackingLog Powershell使用
- Content Authoring with MP4Box
- Content Syndication with RSS
- Parsing XML Files with PowerShell
- Automating Daily Checks with Powershell
- Powershell with file and directory
- Get-Dbf through Powershell 使用Powershell读取DBF数据
- get web url content length
- Get Html Content use HttpDownloader
- [WPF] RichTextBox get/clear content
- vfork()的若干问题
- /bin/basename is needed by jdk
- 显示验证码时,错觉的firefox保存:误以为验证码保存的是当前显示的,其实是重新请求再转存文件.
- 关于谷歌眼镜的几点常见问题
- OnFocusChangeListener监听没反应
- Powershell: Get Content Faster with ReadCount!
- mysql in函数结果不同预期,异常 null
- android 返回键退出
- 重新写cp命令
- android多媒体框架之流媒体AHandler消息机制----base on jellybean(十)
- 用jquery进行表单验证
- js中setInterval与setTimeout用法
- 关于VC++中CMAP用法
- 编程模式: 回调