Managing voluminous amounts of data can be an uphill task for data managers, mainly if your query or scan results run into multiple pages. Pagination in DynamoDB enables the database to handle the large data quantities by breaking the results into multiple manageable pages.This write-up explains the DynamoDB pagination and provides various possible use cases and examples. It also highlights how the pagination in DynamoDB differs from the pagination in other databases.

What Is Pagination in DynamoDB?

Generally, pagination, derived from the word pages, is a technique that is used by databases to split the data records into multiple chunks, segments, or pages. And since AWS DynamoDB supports storing of large amounts of data, it features reliable pagination capabilities.

The DynamoDB pagination component ensures that you can only retrieve up to 1 GB of data per scan or query. While that’s a default setting, you can add a limit parameter in a query to specify a limit. You can further set a limit for the number of records in each scan query.

Notably, a few differences exist between pagination in DynamoDB and pagination in a typical SQL database. Most obviously, each paginated record that is retrieved in DynamoDB comes with a direct cost, making this an unwritten rule when using the pagination in DynamoDB. This feature makes pagination a vital factor in limiting both the retrieved records and direct costs.

How to Use Pagination in DynamoDB

1. Pagination During a Query Operation

In DynamoDB, a query only returns the results of up to 1 MB. But you can effectively confirm if there are more results by scrutinizing your results. Notably, a low-level query operation result contains a LastEvaluatedKey element which is non-null to indicate that there are more items related to your query that you should retrieve.

A result without a LastEvaluatedKey element which is non-null, implies that all the items that match the query fit within the 1 MB limit and there are no more items for retrieval. Of course, you can also set a limit for the number of items per result. See the following sample command:

You can use the previous command to query your table for the items with the same key condition expression values. Let us search our “Orders” table for order_Ids from Darry Tech. We also set a limit to 10 items per page. Another option for the –limit parameter is to use the –page-size parameter for the same purpose.

Pagination is an automatic operation in AWS CLI for items below 1MB of data. You may add an exclusive start key to the command if you want your query to start from a particular order.

The response looks like this:

The provided results show 10 Darry Tech on the first page. You can use the LastEvaluatedKey values to get more orders that match the expression key values of your search to construct a new query. The new query request contains the LastEvaluatedKey values in the ExclusiveStartKey parameter.

An example of the syntax is shown in the following:

The previous command produces the next setoff orders in the next page, starting with the order ID that has the specified primary key, i.e. {“PartitionKey”:{“S”: Darry Tech”},”SortKey”:{“S”:”5356-sy”}}.

2. Pagination During Scan Operations

It is also possible to use the pagination for scan operations. Everything works the same way as with the query commands. However, you need to use the filter-expression attribute. The command looks like what we have here:

The previous command retireves up to 20 items per page from the MyTable table, starting with the item whose primary key is {“PartitionKey”: “ABC123”, “SortKey”: “XYZ987”}. It filters the results to include only the items where the AttributeName attribute has the “ABC123” value.

In the response, the LastEvaluatedKey field contains the primary key of the last item in the result set. You can use this value as the ExclusiveStartKey in a subsequent scan operation to retrieve the next page of results.

Conclusion

Pagination in DynamoDB improves the manageability of data. However, it is vital to know whether your systems will benefit from pagination. It is necessary to use pagination if you have a long list of items in an application. While the provided illustration focuses on the AWS CLI call, you can also use pagination with AWS SDKs such as Python’s Boto3 or any SDK that you prefer.