What Is An Octet Stream

What Is An Octet Stream 5,5/10 8503 votes

Join GitHub today

  1. What Is An Octet Stream

GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.

Sign up

(RFC 2045 and 2046 published November 1996, subtype last updated April November 1996) The 'octet-stream' subtype is used to indicate that a body contains arbitrary binary data. The set of currently defined parameters is: (1) TYPE -- the general type or category of binary data.

Stream New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comments

commented May 2, 2017

My understanding is that the Java SDK sets by default a Content-Type: application/octet-stream header if none is provided by the user when adding an object.
https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L4147

On the other hand, I've noticed that files uploaded via AWS CLI's s3api put-object .. get assigned a Content-Type of binary/octet-stream.
I suppose that this value is assigned by the S3 backend, because I couldn't find any Content-Type header set in the PUT request according to --debug logs. Free ozone 8.

Uploading a file via the S3 Web console produces a binary/octet-stream MIME type as well.
(For the record, Google Cloud Storage also uses binary/octet-stream.)

I was wondering if you had any comment on this difference between application/octet-stream and binary/octet-stream?

referenced this issue May 2, 2017

Merged

[MODE-2692] Return null MIME type in case of 'application/octet-stream' #1656

commented May 2, 2017

What Is An Octet Stream

Hi don't believe the CLI actually sets binary/octet-stream. You can verify this by issuing a command like aws s3 cp MY_BINARY_FILE s3://MY_BUCKET --debug and see that it does not send a Content-Type header.

Per the S3 docs, the default is binary/octet-stream if the header is not set: http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html

commented May 2, 2017

Oops sorry looks like I didn't read your question close enough! I'm not sure what the exact reason is for the discrepancy but I suspect it's because application/octet-stream is registered with the IANA, and binary/octet-stream does not seem to be. In any case, I don't think it's possible for us to change this now because it would be subtle breaking change for customers.

commented May 3, 2017

Octet

Hum, if the default behavior can't be changed (what kind of 'subtle change' do you expect by the way?), what about an option to specify the default MIME type?
Or is the only solution for the API user to explicitly set binary/octet-stream for each PUT operation?

commented May 3, 2017

As far as breaking change, customers may be relying on the fact that the default MIME for objects uploaded using the Java SDK to be application/octet-stream rather than binary/octet-stream.

Do you expect all of your objects to have binary/octet-stream? If so, you can use ClientConfiguration#withHeader. Note that this will override the value set on ObjectMetadata#setContentType so it won't work if the you need to be able to override the default value.

Another option might be a custom RequestHandler

We can also look into adding a default object MIME if that would be easiest for you.

commented May 3, 2017

Thanks for the tips on the various methods available.
I have full control of the code that uses the API so I can explicitly use ObjectMetadata#setContentType for each call, that's not a big deal.
Yet, I am curious what you meant in your last sentence.

commented May 3, 2017
edited

Are you referring to

We can also look into adding a default object MIME if that would be easiest for you.

?

I was imagining just a new option like setDefaultObjectMimeType on S3ClientOptions that works sort of like ClientConfiguration#withHeader but at the S3 level so you can change mimetype per PUT call.

commented May 3, 2017

Yes, I think such an option would be useful. That would even give some API visibility to the fact that the Java SDK uses a 'non-standard' default MIME type.

commented May 4, 2017

By the way, speaking of 'application/octet-stream' header, I saw in debug traces that (all?) HTTP requests sent to S3 contain such a header, even in case of a GET or HEAD.
I suppose that's not forbidden per se according to the spec, but I find it strange.

assigned dagnirMay 15, 2017

referenced this issue Jan 13, 2019

Open

Crash when guess_extension() returns None #37

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Tidal.com streams flac files, but it does not provide with file download, which is what I want. As you would expect from a big company such as Tidal, upon my inquiry, its support service in an utmost polite and friendly manner advised me to go funk myself.

Preferring not to, I learned that when flac track is streamed, several .bin files are being cached in my browser. I saved one of such .bin files - its size approximately corresponds to that of flac file, but I was not able to open it with neither 7Zip nor UltraISO nor by renaming it into zip or flac and opening it with VLC or PotPlayer.

So I guess my question is - can I decode such file or obtain flac from it by some other means? Thanks.

user498283user498283

1 Answer

A quick Google tells me..

'The FLAC stream is encrypted, though it is possible to find the encryption key in a JSON file that the Tidal API pulls right before streaming a song including the direct song URL and as well as the key, but I’m not sure of the algorithm used. I’ve tested many times with OpenSSL to decrypt the files with AES-128 and 256. Either I’ve setup OpenSSL wrong, or they’re using Salsa20 to encrypt the streams. The API calls are going over HTTPS while the streams are not.'

— in short, you're not going to do it.

TetsujinTetsujin

Not the answer you're looking for? Browse other questions tagged conversionflacbinary-filesdecoding or ask your own question.

Posted on