Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> The only downside is that it’s conceptually more complicated, and requires some understanding of underlying components (zip files, http responses, streams).

There's at least one more downside: the user loses all indication of progress as the Content-Length is unknown when the headers are sent



Nice catch, We are working on it!

dumply knows the exact size of each image as it is saved in the DB on upload, and all the zip byte headers are fixed, so the zip file size should be deterministic and calculable even before the first byte is sent. Remember we don't compress the already compressed images.

If you didn't know the file sizes, for example you had raw unknown input streams, or had compressible data, you can still guesstimate the content-length so the user got some progress bar, even if it wasn't 100% accurate.


The IEInternals blog covers each browser's behavior for underrunning the purported Content-Length of a download: http://blogs.msdn.com/b/ieinternals/archive/2011/03/09/brows...

Looks like using browser sniffing, you can deliver an exaggerated Content-Length to everyone but Opera and browsers will deal with it gracefully. Pretty neat. (Obviously not desirable for violating the HTTP spec, but the UX gains might be worth it.)


You're not always dealing with the client that is specified in the UA string. Clients can use proxies, including transparent proxies.

For example major mobile operators pipe HTTP connections through a proxy that recompresses images. In that case you see e.g. Safari's or Opera's UA string, but you're actually dealing with proxy's HTTP behavior.


In dump.ly's use case, they would probably want client-side detection in JavaScript, not the UA string. You definitely have to be conservative implementing such an unexpected feature.

An older article (2008) that also talks about misreporting content-length for fun and profit: http://tech.hickorywind.org/articles/2008/05/23/content-leng...


Client-side detection in JavaScript or UA string aren't mutually exclusive, since the UA string is exposed on the DOM.

Are suggesting they would rather guess the browser by inference through presence of DOM properties and methods?

In the end it narrows down to: would you rather a) break the functionality for some users in exchange to give the best possible solution to others, or b) give an OK experience to everyone

I usually go with "b". I think that frustration is much more powerful than awe.


Wouldn't guestimation cause problems (on user's end)? I don't know, but how browsers/curl/wget/crawlers/... react if you tell them Content-Length is 1000 bytes, and then send them 900 bytes and close the connection? or overshot and send 1100? I have a feeling that they wouldn't like it, or at least it varies between fetcher library implementations.


RogerE states above that under-estimating does lead to resource truncation, and over-estimating leads to the browser waiting until timeout/connection break in case there's more data to be fetched.

But yeah, it's probably indeterminate behavior.


You can do a solid best-guess estimate, especially if those images are already in compressed formats.


You can guess, but browsers are quite picky about this header --- guess any amount too low and the download is truncated. Guess too high, and the browser will wait for an amount of time just to be sure the server isn't going to send more data.


I wonder if this would actually be useful to support in HTTP? (An `Estimated-Content-Length` header or similar, only valid with `Connection: close`.)


It would be better to include progress estimation information as a chunked encoding extension. This allows much better progress reporting.


Note that it is possible to "always guess long" and pad out with null bytes... this usually works (many file format parsers aren't too picky). However, this is more of a practical work-around than a recommended solution!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: