Mac Intel wierdness?

[corey]

I may be doing something wrong here, as s3sync and ruby and my mac are fairly recent additions to my life.  s3sync appears to be skipping the upload step on my macbook (intel).  Identical code on my ubuntu box appears to upload.  Could it be missing a process for step or something?  Am I missing something?  Maybe this 1.8.2 ruby version is the culprit?

on mac: 
 ruby 1.8.2 (2004-12-25) [universal-darwin8.0]

on ubuntu:
 ruby 1.8.4 (2005-12-24) [i486-linux]


Here's my s3sync call (within a bash script):

# run test
./s3sync.rb -n -v -d --delete test/ cds_chf_listing_images_test:test/

Here's the output on mac:
----------------------
testing s3.
s3Prefix test
localPrefix /Users/corey/work/chf-s3/bin/test/
localTreeRecurse /Users/corey/work/chf-s3/bin/test
Test /Users/corey/work/chf-s3/bin/test/16911-1.jpg
Test /Users/corey/work/chf-s3/bin/test/16911-2.jpg
Test /Users/corey/work/chf-s3/bin/test/16911-3.jpg
Test /Users/corey/work/chf-s3/bin/test/17111-1.jpg
Test /Users/corey/work/chf-s3/bin/test/7211-1.jpg
s3TreeRecurse cds_chf_listing_images_test test
Trying command list_bucket cds_chf_listing_images_test max-keys 200 prefix test delimiter / with 100 retries left
Response code: 200
----------------------

Here's the output on ubuntu:
----------------------
testing s3.
s3Prefix test
localPrefix /home/corey/work/chf-s3/bin/test/
localTreeRecurse /home/corey/work/chf-s3/bin/test
Test /home/corey/work/chf-s3/bin/test/16911-1.jpg
Test /home/corey/work/chf-s3/bin/test/16911-2.jpg
Test /home/corey/work/chf-s3/bin/test/16911-3.jpg
Test /home/corey/work/chf-s3/bin/test/17111-1.jpg
Test /home/corey/work/chf-s3/bin/test/7211-1.jpg
local item /home/corey/work/chf-s3/bin/test/16911-1.jpg
local node object init. Name:16911-1.jpg Path:/home/corey/work/chf-s3/bin/test/16911-1.jpg Size:7635 Tag:2b9408c34765b6ac5902beba0b55ae80
s3TreeRecurse cds_chf_listing_images_test test
Trying command list_bucket cds_chf_listing_images_test max-keys 200 prefix test delimiter / with 100 retries left
Response code: 200
source: 16911-1.jpg
s3 node object init. Name:16911-1.jpg Path:test/16911-1.jpg Size: Tag:
Create node 16911-1.jpg
local item /home/corey/work/chf-s3/bin/test/16911-2.jpg
local node object init. Name:16911-2.jpg Path:/home/corey/work/chf-s3/bin/test/16911-2.jpg Size:7970 Tag:f7117281dc41f9d787d92dd4fc03a39a
source: 16911-2.jpg
s3 node object init. Name:16911-2.jpg Path:test/16911-2.jpg Size: Tag:
Create node 16911-2.jpg
local item /home/corey/work/chf-s3/bin/test/16911-3.jpg
local node object init. Name:16911-3.jpg Path:/home/corey/work/chf-s3/bin/test/16911-3.jpg Size:9952 Tag:6af98760f69416f9efce663d5b99df14
source: 16911-3.jpg
s3 node object init. Name:16911-3.jpg Path:test/16911-3.jpg Size: Tag:
Create node 16911-3.jpg
local item /home/corey/work/chf-s3/bin/test/17111-1.jpg
local node object init. Name:17111-1.jpg Path:/home/corey/work/chf-s3/bin/test/17111-1.jpg Size:8310 Tag:30d4590a1ff4065f5f555c19688ff71d
source: 17111-1.jpg
s3 node object init. Name:17111-1.jpg Path:test/17111-1.jpg Size: Tag:
Create node 17111-1.jpg
local item /home/corey/work/chf-s3/bin/test/7211-1.jpg
local node object init. Name:7211-1.jpg Path:/home/corey/work/chf-s3/bin/test/7211-1.jpg Size:9488 Tag:98e0c934f0d539957492ba982a5aad15
source: 7211-1.jpg
s3 node object init. Name:7211-1.jpg Path:test/7211-1.jpg Size: Tag:
Create node 7211-1.jpg
----------------------

[lowflyinghawk]

greg:  there is some conditional code in S3.rb that tests for ruby version < 1.8.3, but I forget if that's what you are using.

[ferrix]

Read readme please.  1.8.2 cannot work; lacks streaming.

[lowflyinghawk]

I agree about 1.8.2 being the wrong thing to use, but isn't streaming something you had to add in by modifying Net::HTTP or one of its subclasses?  I did it like this in S3.rb instead:
        if req.request_body_permitted?  # post/put
          if data.respond_to?(:read)    # file or file-like?

            # setting body_stream and calling http.request with the
            # second argument nil causes the put to stream from the
            # file instead of having to load the whole thing in memory

            req.body_stream = data
            result = @http.request(req,nil)
            # ...

[ferrix]

I don't mean adding streaming to S3 ruby lib, everyone's done that by now .  As you sort of allude, I think you really do need to add some code to net/http to get the *capability* of streaming at all.  And I don't know how easy it is to backport.

I am not even going to consider anything pre-1.8.4 as supported, since that was the minimum version I wrote the original code to!  If someone wants to back-port, I will be happy to maintain a "compatibility" release here.

As background: is mac not capable of easily running anything less archaic than .2?

[lowflyinghawk]

this is the first thing that popped up by googling
http://www.google.com/search?q=ruby+1.8.4+mac&start=0&ie=utf-8&oe=utf-8&client=firefox-a&rls=org.mozilla:en-US:official

 http://wincent.com/knowledge-base/Building_and_installing_Ruby_1.8.4_on_Mac_OS_X_Tiger
it looks pretty simple to me, but I'm used to building stuff on linux.

[corey]

Great, thanks for the quick responses.  (And the great tool.)  I haven't gotten as far as installing make etc on my mac yet, so I'm just doing my development on the linux box.  As far as I can tell, this isn't a problem with the mac-ness or intel-ness, but rather the ruby version that came on my OS.  If I have any trouble after upgrading my ruby I'll repost.