amazon web services - Picking random 100 files in aws s3 path with subdirectories and then copying them to another folder

Question

I have a aws s3 bucket s3://bucket1/ that has multiple subdirectories at two levels with the structure s3://bucket1/shard-*/task-*/filename1.txt and I have to select random 100 files from the bucket. What I did was to use "aws s3 ls recursive" along with awk and shuf to get a list of these 100 files and store it in a file. But now I wanted to copy these 100 files into a specific bucket in s3. How do I go about doing this? Is there a "aws s3 cp" where I can use the list?

question from:https://stackoverflow.com/questions/66052928/picking-random-100-files-in-aws-s3-path-with-subdirectories-and-then-copying-the

Answer 1

Waitting for answers