The Tale of Amazon Direct
Amazon Instant is dead. Long live Amazon Direct.
This is a long tale of intrigue and woe that has nothing to do with film reviewing and probably isn’t of interest to anyone, so I’ve hidden it behind a “read more” link.
We had a fair number of titles on Amazon Instant, which is now discontinued and rolled into Amazon Direct and/or Prime. It isn’t a priority for us, really, because direct sales far outnumber sales through Amazon for the simple reason that we can offer a lower price selling directly, but I finally got around to migrating the videos. Or, at least, attempting to migrate one of them to test the waters: the very short but popular The Last Cartridge (1908).
Let’s preface this with a brief autobiography. Once upon a time, I was a programmer. I did web-based applications in the early 2000s. I got out of it when I decided that I hated the Internet and only begrudgingly accepted computers. Maybe that’s an exaggeration, but not much of one.
Amazon wants closed captioning. Whatever; it’s silly, but I can very easily supply a SRT file containing only the phrase “[no audio]” that lasts the duration of the film. They don’t like the aspect ratio. I understand; 1440×1080 is the redheaded stepchild of HD video. That’s fine, it’s no great trouble to pillarbox it to 1920×1080 so the 4:3 video plays centered in a 16:9 field. They don’t like the framerate. It needs to be one of the modern standards like 29.97fps. All right. I can apply a pull down pattern to bring it up from 16fps. The motion will be jerkier, but that can’t be helped. (FYI: 29.97 is an approximation. The standard NTSC framerate is 30000/1001. That’s only very slightly faster, but it adds up and if you use exactly 29.97, you’re going to get audio desync issues over time. ~The more you know~)
All of that is going to increase the file size, but what’s really going to bloat it is the insistence on using a very high bitrate. It’s obvious Amazon intends to transcode the video themselves, which will diminish the image quality, but with a high enough quality video to start with, it shouldn’t matter. You have to do the same thing when uploading to YouTube (although YouTube knows how to handle 4:3 video, oddball framerates, and pretty much any format/container/compression scheme you can think of).
The Last Cartridge is a bit under ten minutes, and with the restrictions above, the file is 1.23 GB. Now, I don’t exactly live in a metropolitan area with high-speed fiber. I’ve got ADSL and I’m lucky to have that. My upload speed tops out at about 180 KB/s. When I have to upload large files like that to someplace like YouTube, I tend to start them at night and they’ll usually be done by morning. Trouble is, Amazon Direct appears to timeout after ten minutes and the upload fails. I say appears to because, Amazon being Amazon, they’re unfamiliar with the idea of informative error messages and throw up the same generic statement for everything. My guess is that they only tested locally and didn’t consider that real-life uploads would take more than a matter of seconds. Anyway, you can also transfer files from an Amazon S3 bucket. Uploading to S3 only fails maybe half the time (“due to errors”, of course — this is still Amazon), so those are better odds. And while the process may be convoluted, transferring files from S3 to Direct at least semi-works.
Amazon is now happy with the video, happy with the audio, happy with the metadata, schizophrenic in regard to the captions (in one place saying they’re fine, in another not even recognizing they exist) but happy enough to let me proceed. I’m pleased that, as Amazon struggles go, this one was rather painless. It was certainly easier than implementing Amazon Payments, where the API has next to no documentation and what is documented is defined a dozen contradictory ways in a dozen different places and every one of them is just flat-out wrong — it took me a few hours a day for a solid week to puzzle out how it’s supposed to work and get it running with any degree of reliability.
Then, the next day, there’s a problem. The video and audio streams of the MP4 I uploaded are apparently of different length and Amazon is freaking out. There’s no reason for that to matter, it’s well within specification for the audio portion of an MP4 to be shorter than the video, but I’ve long been in the habit of padding it with silence since ancient versions of VLC would loop the audio if it ended before the video, so I’d be quite surprised to find it significantly off. Let’s examine the file: the video stream is 9 minutes and 34.063 seconds. The audio stream is — wait for it — 9 minutes and 34.063 seconds. Really, Amazon. I’m beginning to think your YouTube killer isn’t very well put together.
Did you know you can’t rename folders on S3? That’s weird since S3 has no actual hierarchical file structure that might (might) complicate renaming — the folders are purely for grouping. You’ll need to copy the file from the old folder, paste it into the new one, then delete the original. Oh, but with large files, copying occasionally quietly fails without giving an error. You’ll have to download the new file and make sure it’s still the same. Bear in mind, you’re paying for all this. Yes, S3 has a free trial period, but it’s that uniquely Amazon sort of free where they charge you the moment you register.
I say all this because to transfer a file from S3 to Direct, the file has to be uploaded to a specially named folder that Direct doesn’t tell you until the last second. Then you assign what Direct calls permission and S3 calls bucket policy (you’re left on your own to figure that out) to allow Direct to access the folder. After you do that, it sometimes works and other times it fails with “Access to file was denied”. For Amazon, that’s an amazingly descriptive error message and I must applaud them. I can find no way at all to resolve that since I can’t seem to reload the file and Direct can only ever deal with a single folder at a time, so I have to tell it to forget the S3 bucket and start over again with the file in a new folder.
Repeat ad nauseum. And I’m well nauseo now, I’m afraid. I may return to Amazon at some point, but I need a break. Maybe next week. Or month.
We return to our tale about six weeks later. Let’s forget about our past troubles and try a different film, Across the Mexican Line. Let’s also ignore Amazon’s instructions and instead second-guess what they’re actually doing behind the scenes. I’m going to hazard that most of the Amazon Direct code base was inherited from CreateSpace. Now, CreateSpace started life as CustomFlix (I think that was the name — I’m just going from memory) and they produced on-demand DVDs. Later, after Amazon bought them out, CreateSpace became the easy way to upload to Amazon Video, which at some point was rebranded Amazon Instant (or maybe it was the other way around). The upload process for CreateSpace was a bit unusual: You had to prepare the video as if you were authoring a menu-less DVD, then upload the disc image of that imaginary DVD, from which they’d then rip the video. That’s almost magically stupid, but understandable given CreateSpace’s origins, and hey, it worked.
What format is DVD video encoded in? MPEG-2. Let’s take the exact same video Amazon rejected as an MP4 (the modern standard for web video) and re-encode it as an MPEG-2 (a twenty year old compression scheme designed for optical discs), and whaddaya know, they accept it. Across the Mexican Line is now live on Amazon Prime.
There’s just one tiny problem I’ve run into while migrating over more of the back catalogue: I still have to go through S3 to successfully upload. S3 will auto log-out after however many minutes of inactivity, and for whatever reason, uploading doesn’t count as activity. Further, you can’t resume a partial upload — Amazon simply forgets about it and you have to start over from square one. That’s not something I can’t work around — I just wrote a quick program that fiddles about with the tabs until the upload is complete, because flipping between tabs does count as activity — but I shouldn’t have to.
What’s the point of this story? I don’t know — I hate the Internet and only begrudgingly accept computers. Also, Across the Mexican Line (1911), The Last Cartridge (1908), The Egyptian Mummy (1914), and Cabman Kate (1915) are now on Prime. More will be available at some point.