Home » Developers » Shapeways API » submitting a 16MB or larger STL file
Search Search  
Show: Today's Messages    Show Polls    Message Navigator
submitting a 16MB or larger STL file [message #25938] Mon, 11 April 2011 19:11 UTC Go to next message
avatar mctrivia  is currently offline mctrivia
Messages: 808
Registered: September 2010
Go to my shop
Senior Member
Every time I try to use the api to submit a file under 16MB it works. If I try a file over 16MB it fails and through no error codes just stops running. Anyone else having this problem?


Follow me on twitter http://twitter.com/mctrivia or my blog at http://4ddice.blogspot.com/
Re: submitting a 16MB or larger STL file [message #25986 is a reply to message #25938 ] Tue, 12 April 2011 21:12 UTC Go to previous messageGo to next message
avatar horst  is currently offline horst
Messages: 92
Registered: November 2010
Go to all my models
Member
I work here
This has got to be a glitch since it's not by design; our model size limit is much larger than this.

Our developers are going to look into this problem right away.

I am interested to hear if others are experiencing the problem too, to see whether it's isolated and might somehow have a different cause.

-Ben


Shapeways Front-End Product Lead
Re: submitting a 16MB or larger STL file [message #28349 is a reply to message #25938 ] Mon, 30 May 2011 09:05 UTC Go to previous messageGo to next message
avatar hans.lambermont  is currently offline hans.lambermont
Messages: 48
Registered: June 2010
Go to all my models
Member
I work here
Try to use a single model file encode, (the double encode is also still supported), this backwards compatible API fix should allow larger file uploads.
Re: submitting a 16MB or larger STL file [message #28350 is a reply to message #28349 ] Mon, 30 May 2011 09:17 UTC Go to previous messageGo to next message
avatar icc  is currently offline icc
Messages: 1
Registered: May 2011
Go to my shop
Junior Member
No Message Body

[Updated on: Mon, 30 May 2011 09:19 UTC]

Re: submitting a 16MB or larger STL file [message #28351 is a reply to message #28350 ] Mon, 30 May 2011 09:20 UTC Go to previous messageGo to next message
avatar mctrivia  is currently offline mctrivia
Messages: 808
Registered: September 2010
Go to my shop
Senior Member
i am currently using:
$fileHandle = file_get_contents($dir . $fileName);

            $model = array ('title'     => $name,
                            'file'      => base64_encode($fileHandle),
                            'filename'  => $fileName, ...



not sure what you would have me change. I have successfully uploaded 19MB files. but there is no errors on failure just hangs the script.

I have written in code to auto detect script hangs but it takes up to an hour to detect the hang.


Follow me on twitter http://twitter.com/mctrivia or my blog at http://4ddice.blogspot.com/
Re: submitting a 16MB or larger STL file [message #28352 is a reply to message #28351 ] Mon, 30 May 2011 09:30 UTC Go to previous messageGo to next message
avatar hans.lambermont  is currently offline hans.lambermont
Messages: 48
Registered: June 2010
Go to all my models
Member
I work here
mctrivia wrote on Mon, 30 May 2011 09:20

i am currently using:
$fileHandle = file_get_contents($dir . $fileName);

            $model = array ('title'     => $name,
                            'file'      => base64_encode($fileHandle),
                            'filename'  => $fileName, ...


not sure what you would have me change.


This :

'file' => $fileHandle,

The API mentions base64_encode but does not expect the user to do it. That's the tricky part and the reason why lots of ppl. including our developers do this wrong. The PHP SOAP API does the encode itself already.

-- Hans
Re: submitting a 16MB or larger STL file [message #28353 is a reply to message #28352 ] Mon, 30 May 2011 09:45 UTC Go to previous messageGo to next message
avatar mctrivia  is currently offline mctrivia
Messages: 808
Registered: September 2010
Go to my shop
Senior Member
thanks I will make that change imedietly.

Now if only I could figure out why as soon as I add a material restriction to the upload it fails. trying to add
'materials' => array(63),

to set to ceramic only but it fails the upload tests.


Follow me on twitter http://twitter.com/mctrivia or my blog at http://4ddice.blogspot.com/
Re: submitting a 16MB or larger STL file [message #28356 is a reply to message #28353 ] Mon, 30 May 2011 11:32 UTC Go to previous messageGo to next message
avatar barrys  is currently offline barrys
Messages: 116
Registered: May 2010
Go to all my models
Senior Member
I work here
mctrivia wrote on Mon, 30 May 2011 09:45


Now if only I could figure out why as soon as I add a material restriction to the upload it fails. trying to add
'materials' => array(63),

to set to ceramic only but it fails the upload tests.


You should use the material name instead:

'materials' => array('Glazed Ceramics'),

should do the trick. Note the material name does need to be an exact match.
Re: submitting a 16MB or larger STL file [message #28357 is a reply to message #28356 ] Mon, 30 May 2011 11:38 UTC Go to previous messageGo to next message
avatar mctrivia  is currently offline mctrivia
Messages: 808
Registered: September 2010
Go to my shop
Senior Member
thanks will try that in about an hour. Don't know why I just assumed it would be by id not name.


Follow me on twitter http://twitter.com/mctrivia or my blog at http://4ddice.blogspot.com/
Re: submitting a 16MB or larger STL file [message #28394 is a reply to message #28357 ] Tue, 31 May 2011 03:29 UTC Go to previous messageGo to next message
avatar mctrivia  is currently offline mctrivia
Messages: 808
Registered: September 2010
Go to my shop
Senior Member
all seems to be working so far. Here is my beta release of my first Instant Co-Create Model. I will be releasing the locks on the other fields once I have done some more testing and have successfully restricted any inputs that may cause to hang. System auto detects a crashed Damon after 1 hour and restarts but that is a long delay.

http://icc.propmodule.com/random%20dot%20cup.html


Now if only there was a way I could integrate the form into your page so it could be on a co-create model instead of an outside link.


Follow me on twitter http://twitter.com/mctrivia or my blog at http://4ddice.blogspot.com/
Re: submitting a 16MB or larger STL file [message #28401 is a reply to message #28394 ] Tue, 31 May 2011 07:43 UTC Go to previous messageGo to next message
avatar hans.lambermont  is currently offline hans.lambermont
Messages: 48
Registered: June 2010
Go to all my models
Member
I work here
Just tried your link, cool it works Smile I've 'designed' my own mug !
Re: submitting a 16MB or larger STL file [message #28417 is a reply to message #28401 ] Tue, 31 May 2011 12:21 UTC Go to previous messageGo to next message
avatar mctrivia  is currently offline mctrivia
Messages: 808
Registered: September 2010
Go to my shop
Senior Member
thanks. I have some even cooler designs I plan to get up this way soon.

At present I have to use curl to poll shapeways to find out when you are done processing the model(step 3 processing). Would be nice to be able to do this through API and even be able to get rendered image through API.

Ideally instead of polling I could submit a callback URL so you can tell me when it is done. This would drop bandwidth a air bit as presently I am polling your site every 5 seconds until the processing is done.


Follow me on twitter http://twitter.com/mctrivia or my blog at http://4ddice.blogspot.com/
Re: submitting a 16MB or larger STL file [message #28418 is a reply to message #28417 ] Tue, 31 May 2011 12:51 UTC Go to previous messageGo to next message
avatar hans.lambermont  is currently offline hans.lambermont
Messages: 48
Registered: June 2010
Go to all my models
Member
I work here
mctrivia wrote on Tue, 31 May 2011 12:21

Ideally instead of polling I could submit a callback URL so you can tell me when it is done. This would drop bandwidth a air bit as presently I am polling your site every 5 seconds until the processing is done.


Interesting idea. What do you propose as fallback when this callback URL connection attempt fails ?
Re: submitting a 16MB or larger STL file [message #28424 is a reply to message #28418 ] Tue, 31 May 2011 15:08 UTC Go to previous messageGo to next message
avatar mctrivia  is currently offline mctrivia
Messages: 808
Registered: September 2010
Go to my shop
Senior Member
i would try twice then give up. If 2 attempts fail then my server has likely crashed(happened 3 times in 3 years). I can always retry polling if it has been more then 15min.

for record keeping it would be nice if the callback url would be called for the following

processing complete
model sold
model shipped
model order canceled do to not printable


i could pass you a url such as http://icc.propmodule.com/callback/$id/$error and you could replace $... with apropriate info

This way server can handle things automatically instead of sending me a tone of emails and me having to write a pop3 server to handle them


Follow me on twitter http://twitter.com/mctrivia or my blog at http://4ddice.blogspot.com/
Re: submitting a 16MB or larger STL file [message #36274 is a reply to message #25938 ] Sun, 09 October 2011 21:25 UTC Go to previous messageGo to next message
avatar woody64  is currently offline woody64
Messages: 371
Registered: November 2008
Go to my shop
Senior Member
Yes, my current implementtaion shows the same.
Smaller files are submitted, larger ones stop without a further remark:
I get this:
string(20) "SoapAPI::wsdlconnect" string(28) "Connection successfully made" string(14) "SoapAPI::login" string(57) "we got the following session : 1ujpr06jvduamm6tk1e6e6oc05" string(10) "uploadFile"

Additionally I run in memory problems of my php installation. It's hosted on a provider and reports memory allocation failures wih larger files.

Woody64
Re: submitting a 16MB or larger STL file [message #36277 is a reply to message #36274 ] Sun, 09 October 2011 21:40 UTC Go to previous messageGo to next message
avatar mctrivia  is currently offline mctrivia
Messages: 808
Registered: September 2010
Go to my shop
Senior Member
I use 512MB of RAM for my render script and 256MB for upload to make sure it is always shapeways causing the failure. It sucks that shapeways abandend the API before finishing it or fixing bugs like this. If you use binary STL makes it less likely to hit the limit but it is still really easy to hit.


Follow me on twitter http://twitter.com/mctrivia or my blog at http://4ddice.blogspot.com/
Re: submitting a 16MB or larger STL file [message #36280 is a reply to message #36277 ] Sun, 09 October 2011 21:50 UTC Go to previous messageGo to next message
avatar woody64  is currently offline woody64
Messages: 371
Registered: November 2008
Go to my shop
Senior Member
I've already switched to binary stl but I'm running also very quickly into the limit.
Joining more then 5 files and I'm in.

At the end there's also the possibility to zip but that's oncemore an extra step/workaround.

But at least the concept is proven, which makes it also possible to upload per hand.

Woody64

[Updated on: Sun, 09 October 2011 21:52 UTC]

Re: submitting a 16MB or larger STL file [message #36299 is a reply to message #36280 ] Mon, 10 October 2011 04:51 UTC Go to previous messageGo to next message
avatar mctrivia  is currently offline mctrivia
Messages: 808
Registered: September 2010
Go to my shop
Senior Member
i need to start experimenting with zipping on the fly. Some experimenting shows that zipped stl files are about 50% original size. At that compression ratio I am getting a lot closer to the 1,000,000 triangle limit.


Follow me on twitter http://twitter.com/mctrivia or my blog at http://4ddice.blogspot.com/
Re: submitting a 16MB or larger STL file [message #46059 is a reply to message #36299 ] Wed, 28 March 2012 06:42 UTC Go to previous messageGo to next message
avatar abite  is currently offline abite
Messages: 47
Registered: February 2012
Go to my shop
Member
Hi mctrivia.

Is everything OK for submitting files now?
I have the same problem. I had thought my case was exactly same as yours, but I figured out the size of file didn't matter in my case.
If the file size is somewhat large and it takes over 3 minutes(180 seconds), it fails receiving any result or error. However, the file is uploaded. There is no errors on failure just hangs the script.... same as your case.
Less than 3 minutes, everything is perfact.

As hans mentioned, I did change the code like below.
'file' => $fileHandle
but, still in the same situation.

How did you solve the problem? Did you just do what hans said?
I have tried to solve this problem for several days... checked my server, script, my computer...

Any advice would be deeply appreciated. Thanks.

JC

[Updated on: Wed, 28 March 2012 10:24 UTC]

Re: submitting a 16MB or larger STL file [message #46074 is a reply to message #46059 ] Wed, 28 March 2012 12:33 UTC Go to previous messageGo to next message
avatar hans.lambermont  is currently offline hans.lambermont
Messages: 48
Registered: June 2010
Go to all my models
Member
I work here
Hi all, reproduced the issues you showed while the upload itself claims success though was slower than I had expected : it took 18 minutes to upload an STL of 49 MiB. I uploaded from a place with 1.4 Mbit/s max uplink speed, so the file should have been transferred in 5 minutes instead.

The file arrived broken; only 18 bytes :-/ We'll look into that.

-- Hans

[Updated on: Wed, 28 March 2012 13:15 UTC]

Re: submitting a 16MB or larger STL file [message #46079 is a reply to message #46074 ] Wed, 28 March 2012 13:13 UTC Go to previous messageGo to next message
avatar abite  is currently offline abite
Messages: 47
Registered: February 2012
Go to my shop
Member
Dear Hans,

Thank you very much for checking quickly. I have been checking every possibility to solve the problem.
Then, it becomes obvious that the issue is caused by my side.
I can see that the Apache setting of my server indicates
"Timeouts Connection: 180 - Keep-Alive: 3"
It has been the only suspicous factor so far.
I am gioing to talk with my server hosting agent.
Thanks again.

[Updated on: Wed, 28 March 2012 13:18 UTC]

Re: submitting a 16MB or larger STL file [message #47812 is a reply to message #36277 ] Thu, 03 May 2012 13:22 UTC Go to previous message
avatar bdickason  is currently offline bdickason
Messages: 74
Registered: November 2011
Go to my shop
Member
I work here
mctrivia wrote on Sun, 09 October 2011 21:40

I use 512MB of RAM for my render script and 256MB for upload to make sure it is always shapeways causing the failure. It sucks that shapeways abandend the API before finishing it or fixing bugs like this. If you use binary STL makes it less likely to hit the limit but it is still really easy to hit.


I agree it sucks that the API wasn't finished. As such, we're starting to work on it again in a few weeks to turn it into something real Smile As in... proper functions, RESTful, documentation, oAuth, etc.

Don't have a timeline yet but we'll probably release in a heavily iterative cycle and use these forums to get feedback and improve.

We will also continue to support the existing API for quite some time until everyone (fingers crossed) migrates over to the new version Very Happy


P.S. Any examples of sites that do a good job of integrating external code on the site? (Referring to your 'It would be great if I could get this on the product detail page' comment) Obviously Facebook does a good job here but I'm wondering if there are any other examples? This is definitely our vision and we're still working through the best way to make this happen.

Instant co-creators (we just call them Creators) are the future!!!


Director of Product - Shapeways

 
   
Previous Topic:Product appears as not for sale even though it is for sale :o
Next Topic:string(27) "Error Fetching http headers" -- ??