cassandra paging 分页方法应用

来源:互联网 发布:php怎么接私活 编辑:程序博客网 时间:2024/06/14 03:15

费了2天的时间,终于找到了如何实现分页的办法。 cassandra 只在nodejs driver 1.0里提到了如何分页,后面就再也没有提过了,坑爹啊。 网络上搜遍了,都只有java的driver example, 也没有说这个pageState这么用。 

要实现分页,driver execute是做不到的,所以要用eachRow 的API实现分页, 要用到的参数主要是:
autoPage: 是否要连续查询所有页面
pageState: 用来存放最后一次查询最后一条记录的token, 这样下次查询的时候就直接从下一条记录开始查,这个很重要,否则就是网站上看到的其他人贴出来的手动记录方法了
fetchSize: 每一页查询多少记录

对于我们的应用是: 查出100万条记录,然后分页到网站使用。 需要提供总得记录数字,和实时查询需要页数内的记录。
1. 总的记录数目:这个必须要做一次全部查询,目前只有这个办法。 这个API有个好处,就是有一个function(n,row), 当你不做任何操作的时候,他不保存任何记录信息,这样做就可以不需要buffer 来保存每一条具体的记录,而只是在最后的function(err, result) 拿出rowLength 即可; 

2. 分页查询: 假设按webpage 每页25条记录,先给出webpage 1的数据, 那么就是page1 (注意,webpage 是网站上显示的页面, page是后端数据库的page);如果此时客户直接点了webpage3, 跳过了webpage2, 那么对后端其实没有影响,还是照样给page2的信息给前段即可,因为查出来的是根据token排序的,不是primary_key本身,所以对于客户来说,查出来的基本是无序的; 

Fetching large result sets

The single-threaded nature of Node.js should be taken into consideration when dealing with retrieving large amount of rows. If an asynchronous method uses large buffers, it could cause large memory consumption and a poor response time.

Because of this, the driver exposes the #eachRow() and #stream() methods. When using these methods, the driver will parse the rows and yield them to the user as they come through the network. All methods use a default fetchSize of 5000 rows, retrieving only the first page of results up to a maximum of 5000 rows.

Automatic paging¶

If you want to retrieve the next pages, you can do it by setting autoPage: true in the queryOptions to automatically request the next page, once it finished parsing the previous page.
// Imagine a column family with millions of rows.var query =  'SELECT * FROM largetable';client.eachRow(query, [], { autoPage : true }, function(n, row) {   // This function will be called per each of the rows in all the table.    }, callback);

Manual paging¶

If you want to retrieve the next page of results only when you ask for it (that is, a web pager), you use the pageState made available by the end callback in #eachRow().
client.eachRow(query, [], { prepare : 1 , fetchSize : 1000 }, function (n, row) {      // Row callback.     }, function (err, result) {        // End callback.        // Store the paging state.        pageState = result.meta.pageState;     });
In the following kinds of requests, use the page state.
// Use the pageState in the queryOptions to continue where you left it.client.eachRow(query, [], { pageState : pageState, prepare :  1 , fetchSize :  1000 }, function (n, row) {   // Row callback.   }, function (err, result) {      // End callback.      // Store the next paging state.      pageState = result.meta.pageState;   });
http://docs.datastax.com/en/developer/nodejs-driver/1.0/nodejs-driver/reference/paging.html
0 0
原创粉丝点击