Title: Mac Intel wierdness? Post by: corey on May 03, 2007, 11:53:47 PM I may be doing something wrong here, as s3sync and ruby and my mac are fairly recent additions to my life. s3sync appears to be skipping the upload step on my macbook (intel). Identical code on my ubuntu box appears to upload. Could it be missing a process for step or something? Am I missing something? Maybe this 1.8.2 ruby version is the culprit?
on mac: ruby 1.8.2 (2004-12-25) [universal-darwin8.0] on ubuntu: ruby 1.8.4 (2005-12-24) [i486-linux] Here's my s3sync call (within a bash script): # run test ./s3sync.rb -n -v -d --delete test/ cds_chf_listing_images_test:test/ Here's the output on mac: ---------------------- testing s3. s3Prefix test localPrefix /Users/corey/work/chf-s3/bin/test/ localTreeRecurse /Users/corey/work/chf-s3/bin/test Test /Users/corey/work/chf-s3/bin/test/16911-1.jpg Test /Users/corey/work/chf-s3/bin/test/16911-2.jpg Test /Users/corey/work/chf-s3/bin/test/16911-3.jpg Test /Users/corey/work/chf-s3/bin/test/17111-1.jpg Test /Users/corey/work/chf-s3/bin/test/7211-1.jpg s3TreeRecurse cds_chf_listing_images_test test Trying command list_bucket cds_chf_listing_images_test max-keys 200 prefix test delimiter / with 100 retries left Response code: 200 ---------------------- Here's the output on ubuntu: ---------------------- testing s3. s3Prefix test localPrefix /home/corey/work/chf-s3/bin/test/ localTreeRecurse /home/corey/work/chf-s3/bin/test Test /home/corey/work/chf-s3/bin/test/16911-1.jpg Test /home/corey/work/chf-s3/bin/test/16911-2.jpg Test /home/corey/work/chf-s3/bin/test/16911-3.jpg Test /home/corey/work/chf-s3/bin/test/17111-1.jpg Test /home/corey/work/chf-s3/bin/test/7211-1.jpg local item /home/corey/work/chf-s3/bin/test/16911-1.jpg local node object init. Name:16911-1.jpg Path:/home/corey/work/chf-s3/bin/test/16911-1.jpg Size:7635 Tag:2b9408c34765b6ac5902beba0b55ae80 s3TreeRecurse cds_chf_listing_images_test test Trying command list_bucket cds_chf_listing_images_test max-keys 200 prefix test delimiter / with 100 retries left Response code: 200 source: 16911-1.jpg s3 node object init. Name:16911-1.jpg Path:test/16911-1.jpg Size: Tag: Create node 16911-1.jpg local item /home/corey/work/chf-s3/bin/test/16911-2.jpg local node object init. Name:16911-2.jpg Path:/home/corey/work/chf-s3/bin/test/16911-2.jpg Size:7970 Tag:f7117281dc41f9d787d92dd4fc03a39a source: 16911-2.jpg s3 node object init. Name:16911-2.jpg Path:test/16911-2.jpg Size: Tag: Create node 16911-2.jpg local item /home/corey/work/chf-s3/bin/test/16911-3.jpg local node object init. Name:16911-3.jpg Path:/home/corey/work/chf-s3/bin/test/16911-3.jpg Size:9952 Tag:6af98760f69416f9efce663d5b99df14 source: 16911-3.jpg s3 node object init. Name:16911-3.jpg Path:test/16911-3.jpg Size: Tag: Create node 16911-3.jpg local item /home/corey/work/chf-s3/bin/test/17111-1.jpg local node object init. Name:17111-1.jpg Path:/home/corey/work/chf-s3/bin/test/17111-1.jpg Size:8310 Tag:30d4590a1ff4065f5f555c19688ff71d source: 17111-1.jpg s3 node object init. Name:17111-1.jpg Path:test/17111-1.jpg Size: Tag: Create node 17111-1.jpg local item /home/corey/work/chf-s3/bin/test/7211-1.jpg local node object init. Name:7211-1.jpg Path:/home/corey/work/chf-s3/bin/test/7211-1.jpg Size:9488 Tag:98e0c934f0d539957492ba982a5aad15 source: 7211-1.jpg s3 node object init. Name:7211-1.jpg Path:test/7211-1.jpg Size: Tag: Create node 7211-1.jpg ---------------------- Title: Re: Mac Intel wierdness? Post by: lowflyinghawk on May 04, 2007, 06:56:25 PM greg: there is some conditional code in S3.rb that tests for ruby version < 1.8.3, but I forget if that's what you are using.
Title: Re: Mac Intel wierdness? Post by: ferrix on May 04, 2007, 11:11:24 PM Read readme please. 1.8.2 cannot work; lacks streaming.
Title: Re: Mac Intel wierdness? Post by: lowflyinghawk on May 05, 2007, 08:30:24 AM I agree about 1.8.2 being the wrong thing to use, but isn't streaming something you had to add in by modifying Net::HTTP or one of its subclasses? I did it like this in S3.rb instead:
if req.request_body_permitted? # post/put if data.respond_to?(:read) # file or file-like? # setting body_stream and calling http.request with the # second argument nil causes the put to stream from the # file instead of having to load the whole thing in memory req.body_stream = data result = @http.request(req,nil) # ... Title: Re: Mac Intel wierdness? Post by: ferrix on May 05, 2007, 09:45:51 AM I don't mean adding streaming to S3 ruby lib, everyone's done that by now ;). As you sort of allude, I think you really do need to add some code to net/http to get the *capability* of streaming at all. And I don't know how easy it is to backport.
I am not even going to consider anything pre-1.8.4 as supported, since that was the minimum version I wrote the original code to! If someone wants to back-port, I will be happy to maintain a "compatibility" release here. As background: is mac not capable of easily running anything less archaic than .2? Title: Re: Mac Intel wierdness? Post by: lowflyinghawk on May 06, 2007, 06:04:38 AM this is the first thing that popped up by googling
http://www.google.com/search?q=ruby+1.8.4+mac&start=0&ie=utf-8&oe=utf-8&client=firefox-a&rls=org.mozilla:en-US:official http://wincent.com/knowledge-base/Building_and_installing_Ruby_1.8.4_on_Mac_OS_X_Tiger it looks pretty simple to me, but I'm used to building stuff on linux. Title: Re: Mac Intel wierdness? Post by: corey on May 07, 2007, 07:55:54 PM Great, thanks for the quick responses. (And the great tool.) I haven't gotten as far as installing make etc on my mac yet, so I'm just doing my development on the linux box. As far as I can tell, this isn't a problem with the mac-ness or intel-ness, but rather the ruby version that came on my OS. If I have any trouble after upgrading my ruby I'll repost.
|