I am not able to find a solution for delta upload to S3 bucket. Every time its uploading the entire folder.
Is there a provision for uploading only the changed + new files?
Adding EXTRA_ARGS: "--size-only" on the pipeline variable did the work for me..
As per document by AWS, `--size-only (boolean) Makes the size of each key the only criteria used to decide whether to sync from source to destination.`.
Hope this helps.
same issue here ... cant have partial sync whereas everything gets uploaded at any commit
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.