How ChatGPT Works

 https://mitsloanedtech.mit.edu/ai/basics/how-chatgpt-works-a-non-technical-primer/#Watch_the_Video

API Architectural Types

 API types

Aman Shark Tank

 https://youtube.com/clip/UgkxS9MzQ5ZXe8yinaft11ZJOaUvtHv9rqJ9

 

https://www.youtube.com/watch?v=En5uGDtL-MQ

start time 25m 35s

 https://www.youtube.com/watch?v=ddfQLld0uLU

start time : 19m 25s

DynamoDB should i use scan or query ?

Difference Between Query and Scan in DynamoDB

Query and Scan are two operations available in DynamoDB SDK and CLI for fetching a collection of items. While they might seem to serve a similar purpose, the difference between them is vital. While Scan is "scanning" through the whole table looking for elements matching criteria, Query is performing a direct lookup to a selected partition based on primary or secondary partition/hash key. You will also see the difference in speed. While Query usually returns results within 100ms, Scan might even take a few hours to find the relevant piece of data.

When querying DynamoDB, When You Should Use Scan vs Query ?
Generally speaking, you should always favor Query over Scan. When it's not possible (for example, when you're looking for a piece of data with a key that is unknown to you), and if it's a frequently used pattern, consider adding a GSI (Global Secondary Index) to index that attribute and enable Query. In the last resort, use Scan.

I need to use DynamoDB scan. Can I make it faster?

You can consider using DynamoDB Parallel Scan. It uses multiple threads to run multiple scans at once scanning multiple parts of your table space simultaneously.

When To Use DynamoDB Scan

But sometimes using scans is inevitable, you only need to use them sparingly and with knowledge of the consequences. Here are use-cases by scans might make sense: 

• Getting all the items from the table because you want to remove or migrate them
• If your table is really small (< 10 MB) and it takes a few calls to scan through all of it
• Scan operation can be also used on secondary index, especially if we are dealing with sparse index. In this case, scan will return only items that have an attribute which is indexed by the selected index, e.g. deleted entities.

Restricting usage of DynamoDB Scans using AWS IAM

If you don't want to rely on "good will" and want to codify the best practices instead, you can restrict usage of Scans at all for all users in your organization/AWS account using following IAM policy:

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Action": "dynamodb:Scan", "Resource": "*" } ] }

DynamoDB Scan Filtering 

In order to minimize the amount of data returned from the database to the application, you can use FilterExpressions. Filter expressions allow you to select and return only a subset of scan results. 

How fast is DynamoDB scan?

DynamoDB Scan is not a fast operation. Because it goes through the whole table to look for the data, it has O(n) computational complexity. If you need to fetch data fast, use Query or Get operations instead. 

What is the DynamoDB scan cost?

DynamoDB Scan cost depends on the amount of data it scans, not the amount of data it returns. 

Parallel Scan in DynamoDB Scans are generally speaking slow. To make that process faster, you can use a feature called "Parallel Scans" which divide the whole DynamoDB Table into Segments. A separate thread/worker then processes each Segment so N workers can work simultaneously to go through the whole keyspace faster. Creating Parallel Scan is quite easy. Each of your workers, when issuing a Scan request should include two additional parameters: Segment - Number of segments to be scanned by a particular worker Total Segments - Total amount of Segments/Workers/Threads

But, be careful with Parallel scans as they can drain your provisioned read capacity pretty quickly incurring high costs and degrading the performance of your table.