|
Title: S3sync seems to be uploading/checking but s3Fox doesn't show the files Post by: tnik on February 04, 2008, 10:55:47 AM Strange problem here. I have just recently moved thousands of files to our backup server, and I have our backup server using s3sync to amazon. Log file shows that the files were created (eg. Create node A-E/ACME_DYNAMICS/ACM002C.M09) but when I check from s3fox, the files aren't there.
this is the script I'm running.. Code: #!/bin/bash BUCKETPATH=backup-data:/toolcrib/ BACKUPPATH=/backup_data/toolcrib/ EXCLUDE= date echo start toolcrib bu... cd /root/s3sync && /usr/bin/ruby s3sync.rb -rvs ${BACKUPPATH} ${BUCKETPATH} echo done toolcrib bu... date If I were to run it right now in debug mode, here is an clip from debug output. Code: local item /backup_data/toolcrib/A-E/ALABAMA_METALS/24-135 WEAR TRACK SPLICE JOINTS.3d local node object init. Name:A-E/ALABAMA_METALS/24-135 WEAR TRACK SPLICE JOINTS.3d Path:/backup_data/toolcrib/A-E/ALABAMA_METALS/24-135 WEAR TRACK SPLICE JOINTS.3d Size:20176 Tag:4495f29583f73507fc0897f68fb33bce prefix found: /A-E/ALABAMA_METALS/ s3TreeRecurse backup-data /toolcrib /A-E/ALABAMA_METALS/ Trying command list_bucket backup-data max-keys 200 prefix /toolcrib/A-E/ALABAMA_METALS/ delimiter / with 100 retries left Response code: 200 S3 item /toolcrib/A-E/ALABAMA_METALS/24-135 WEAR TRACK SPLICE JOINTS.3d s3 node object init. Name:A-E/ALABAMA_METALS/24-135 WEAR TRACK SPLICE JOINTS.3d Path:/toolcrib/A-E/ALABAMA_METALS/24-135 WEAR TRACK SPLICE JOINTS.3d Size:20176 Tag:4495f29583f73507fc0897f68fb33bce source: A-E/ALABAMA_METALS/24-135 WEAR TRACK SPLICE JOINTS.3d dest: A-E/ALABAMA_METALS/24-135 WEAR TRACK SPLICE JOINTS.3d Node A-E/ALABAMA_METALS/24-135 WEAR TRACK SPLICE JOINTS.3d unchanged Going to my backup server in the 'ALABAMA_METALS' directory, there are 125 files, s3sync shows that those 125 files have been uploaded and unchanged from previous syncs, but s3fox doesn't show anything being in there :-\ any ideas? Title: Re: S3sync seems to be uploading/checking but s3Fox doesn't show the files Post by: tnik on February 04, 2008, 01:38:28 PM ok.. for sh_ts and giggles, I went ahead and tried to do a restore with s3sync, and it created all the files that 's3fox' doesn't show.. verrrry strange.. is there another windows based program that I could use to 'double' check s3, or any info that I should know why s3fox is blind to some files but not others?
Title: Re: S3sync seems to be uploading/checking but s3Fox doesn't show the files Post by: tnik on February 04, 2008, 05:18:20 PM sucks talking to myself, but at least I'm updating ;)
ok, files seem to be transfering fine.. strange that I can't see them in any other client other than using s3cmd list. is there a way to get the listing of the ACL of the files from s3cmd? maybe somehow some files have different permissions setup? I was really hoping I would be able to use s3fox or the like to just grab a file here or there when needed, and not have to goto the server and s3cmd when needed. Title: Re: S3sync seems to be uploading/checking but s3Fox doesn't show the files Post by: ferrix on February 04, 2008, 09:55:47 PM Already loads of threads talking about s3fox incompat with s3sync, and about the fiction of "directories" on s3. I have mixed luck with fox. Sometimes it shows a "dir" as a file, but if you add that to the path manually it will go "in" to the "dir" that it didn't think was there.
Title: Re: S3sync seems to be uploading/checking but s3Fox doesn't show the files Post by: tnik on February 05, 2008, 07:21:46 AM oh I'm sorry, I did search around a bit.. guess I should have dove deeper ;)
yea it's strange, even the 'manual' approach doesn't work when going into the directories, which I guess is just a placeholder.. I did find a win based program, Bucket-Explorer, that you can switch between viewing in 'folder view' or full path view.. doing full path view I can see the mysterious keys/files (sometimes only by entering the absolute path), but as for being in 'folder view' even entering the path manually doesn't jive. and if there is anything I can do to try to finger the culprit on this, just yell.. |