You are viewing jerub

Stephen Thorne's Journal
[Most Recent Entries] [Calendar View] [Friends]

Below are the 20 most recent journal entries recorded in Stephen Thorne's LiveJournal:

    [ << Previous 20 ]
    Monday, November 8th, 2010
    9:02 pm
    Doesn't Anyone Think This Is a Problem?
    I can trivially get root on a machine if the user is member of the 'mock' group.

    $ /usr/bin/mock -r epel-5-i386 --copyin /bin/sh /
    $ /usr/bin/mock -r epel-5-i386 --shell 'chmod 4755 /sh'
    $ /var/lib/mock/epel-5-i386/root/sh

    In order to use mock properly, you have to be in the mock group. It just doesn't work properly even with sudo unless you're in that group. Once you're in that group, you can elevate privs in the chroot it creates and run arbitary scripts.

    I did a quick google and I can't even find anything with keywords like "mock is a gigantic security hole because mock has lots of suid root stuff in it that lets you trivially root a machine if you've ever gotten it to work properly in the past."

    It doesn't make any sense to me. Talking to two admins today led me to conclude that they both knew about this problem, and thought it wasn't an issue.

    Friday, October 29th, 2010
    7:21 pm
    Removing DRM from Adobe ePub File

    I bought a book from an online store a few weeks back. I wasn't aware before I bought the book that I was in fact buying a damaged book. Over the course of the next few days I discovered a number of interesting things about damaged books, some of which other people on the internet are aware of, some of which are very hard to find out. Here are a few notes I made about them.

    First of all. The websites you can use to buy Adobe ePub format books are interesting. They don't provide any content, they are just shops that sell you a reciept for a book that you can then go claim the book from adobe. The receipt is a small xml file that has a file extension that's associated with Adobe Digital Editions. The XML file has an extension '.acsm'.

    At this point you can download the book without ADE, the 'book' is an ePub file. ePub is a .zip file format similar to the ones used by OpenOffice and Microsoft Office. It contians a 'mimetype' file containing the mimetype, a META-INF directory, and a few xml files, and the book encoded in .html with media such as images. All the .html files are encrypted with AES encryption.

    It should be noted here. ADE runs on windows and mac. If you run linux, you're stuck until you run your book through ADE on a windows or mac machine.

    If you download the book too many times you get this message: 'This eBook has already been downloaded the permitted number of times.' So don't fiddle with trying to download the book by extracting the url out of the ACSM file and running tools on it.

    This is where it gets a little complicated, and this seems to be relatively undocumented. Damaged books that you get when you buy them from one of these websites exist in two states. Before ADE and after ADE. Before ADE there will be a pertinent XML files: 'META-INF/encryption.xml'. After ADE there will be another XML file, 'META-INF/rights.xml'. This file is added by ADE and it stores the keys to decrypt the encrypted files. There are secrets involved, spelling them out:

    • Adobe issues you some keys, stored in ADE and in your mobile readers.
    • Adobe knows the keys for the book.
    • Adobe embeds the keys for the book in the book, encrypted with the keys issued to you.

    Saying it another way, Adobe Digital Editions stores on disk the key required for you to open the book you bought in the book itself. In order to read the damaged book, you apply your key issues by adobe to the encrypted data in rights.xml and that results in the key required to decrypt the .html files.

    There are steps published on the net for how to do this decryption in a more useful way, resulting in an undamaged ePub, allowing it to be read in any application that can read epub files. I really like using Aldiko ePub reader on my Android phone, but that can't read Adobe ePub.

    The website 'I <3 Cabbages' has two tools that you may find useful if you have an Adobe ePub book. One tool extracts out of Adobe Digital Editions the keys they issued to you. These keys are yours but they make them very hard to export so you need the weird python script to find them. The second tool is for taking those keys and using them to unencrypt and rezip the .epub file.

    I didn't use the first tool because I used a coworkers windows machine for 15 minutes at lunchtime to download the book onto my phone. I extracted my key off the phone instead of out of ADE. If anyone really wants I can give that process but it's much more effort than it's worth.

    The second tool gave me quite a bit of trouble because I didn't realise there was a difference between the book that ADE put on my phone and the book that I downloaded by inspecting the XML of the '.acsm' file. '' crashed badly because rights.xml (and hence the encrypted AES key for decrypting the .html files) was missing. Whoops. By copying the book back off the phone onto my linux machine I was able to throw it at and get a book I can read on any reader without restrictions.

    Wednesday, June 4th, 2008
    8:36 pm
    So I had a terribly tuesday. Probably my worst day ever. I handled it quite well considering all that went on on tuesday.

    Read more...Collapse )
    Sunday, November 18th, 2007
    1:39 pm
    Person1: I bought this t-shirt for Count Down.
    Me (whispering loudly to person2): What's countdown?
    Person2 (whispering loudly): I believe it is a music program similar to our iTunes of today.
    Me: Really?
    Person2: They had a search function called Molly Meldrum.

    Tuesday, May 29th, 2007
    12:14 pm
    The secret ingredient to make an acceptable lasagne into an awesome lasagne is Kangaroo.
    Sunday, April 1st, 2007
    8:58 am
    It's Blue
    Well, we're pleased to announce that Elspeth's pregenancy test came back and we're going to have a baby! I'm so excited!!

    More news soon.

    April Fools!
    Friday, February 9th, 2007
    7:55 pm
    The dishwasher's back.

    I'm filling out the required paperwork to kick butts right now.
    4:27 pm
    Real, Estate
    Well, that was interesting.

    Just got off the phone to the realestate agent. Apparently 'Barry', the owner of our property, was the person who came into our house while we were no there without giving any notice whatsoever to remove our dishwasher for repair.

    Very unimpressed.
    Friday, September 15th, 2006
    9:15 am
    SFD burning
    Thanks to everyone who helped burn the 100 CDs for Software Freedom Day.

    It was actually interesting watching how we burned on the different operating systems. There must have been a different version of Nero on every windows PC.

    Joel had the best burning process going because it was writing to both his cd burners at 40x at the same time.

    To begin with, on ubuntu, I got elspeth to burn cds using cdrecord -verbose -speed 40 TheOpenCD.iso. But after Ali asked "how do I burn cds" I explored a teensy bit and found there was, "right click on iso, write to cd." And that gui had a nice little button, which was, "Burn another CD". The only problem was that ejecting, reinserting and clicking "burn another cd" was a relatively long process.

    Elspeth's mactop didn't have an especially fast burner, but the swapover process was seamless, because what I did there was while true; do hdiutil burn TheOpenCD.iso; done, which would burn and verify the cd, eject the cd, and await a new one being inserted. No tray either, the laptop had a slotloader.

    Thanks to Alisa, Elspeth, Erica and Ali for helping us stomp all the cds. Thanks to Joel, Erica, Ali, Rohan and Elspeth for burning all those cds. :)
    Tuesday, June 20th, 2006
    8:51 pm
    Monday, May 8th, 2006
    10:04 pm
    Public Speaking
    Public speaking always makes me nervous before and after, but during I feel like I'm flying.

    I think my darcs talk went well.
    Monday, April 3rd, 2006
    4:37 pm
    We Need Someone.
    We need a python developer, here in Brisbane, Australia.

    Here's the position.
    Tuesday, January 24th, 2006
    1:15 pm
    Java^WPython Hacking
    Yeah, so Paul Gearon was a little frustrated with java, and was talking about java, and using java to solve a Problem that he was given by a coworker.

    Restating the Problem here, it is: Take all the characters in a java.lang.String, and return a string containing a single instance of each of those characters, in sorted order.

    I barely read the javaish bits about his post, but I did notice that the length of the solutions was quite long, and being the pythonista that I am, I felt compelled to troll a little.

    The python solution to the Problem quite simple - the one I gave was under 10 lines, including docstring and asserts. Later, I was accused of giving a '2 line' solution, and while that's strictly true, wasn't the case at all. In two lines you can barely call a builtin, let alone make sure that you've got the right python version. I also didn't solve the problem correctly - I gave a solution that returned a list, not a sequence.

    So I wrote, and tested (with the help of #python/freenode) a complete solution that passes tests on 1.5.2, 2.2.3, 2.3.5 and 2.4.1.

    Anyway, I had a little bit of fun doing that. It feels weird, being a person who loves generator expressions and decorators, and can't wait to get his hands on the new generator functions in python2.5 - to write code that would run 7 years ago.

    Of course, the python2.4 solution that I'd be using if I didn't want to present a backwards compatible version is:
    def sortedUniqueString(seq):
        return ''.join(sorted(set(seq)))

    Plus docs and tests, of course!
    Tuesday, December 6th, 2005
    4:20 pm
    Java bashing

    I'm not sure I have to say anything.
    Tuesday, November 22nd, 2005
    2:18 pm
    Monday, October 24th, 2005
    9:50 am
    copy, right?
    from tim o'reiley, via david ascher

    Google's opt out approach is the only way to cut the Gordian knot of forgotten rights and permissions.

    Oh, hell yeah. Wouldn't it be great if we had a list of pieces of music, books, short stories and movies that you weren't allowed to copy, because the copyright holder had denied that right. And the rest you would be able to copy freely, because the copyright holder hadn't declared their right?

    Monday, October 17th, 2005
    8:56 am
    I read today that the debian folks are considering a new source package format, called the 'Wig-and-Pen'.

    It was named after a particular pub that it was designed in, during 2005.

    The Wig and Pen is the source of many happy memories for me, mostly because I drank there most nights when I was in Canberra. This was the result of someone in the little hallway that joined all the lecture theatres together saying, "Sod it, someone said 'The Wig and Pen' and that's where we're going."

    As a result, 80-100 people turn up to this watering hole on a monday night, when there are barely 2 people staffing the place. We come within litres of drinking them out of their homebrew beer, and eat them completely out of food. I was asked quite politely how long the conference is in town, and how many people to expect on a nightly basis.

    Many a quick hack was written on those nights. I believe there was a hurd L4 filesystem that was fixed, I designed most of the solution that would eventually net me a powermac g5 in the hackfest, and, so it seems, there was some debian package design that went on while I wasn't watching.
    Monday, October 10th, 2005
    12:15 pm
    the day has finally come.
    And announcing, the latest in the Non-Evil(tm) technologies, Google Reader!.

    It's an RSS aggregator (your /friends page is a livejournal aggregator, think of google reader as one that can do both lj and the rest of the webbernet) that uses the excessively cool web browser stuff that we've come to expect since gmail made javascript and the web see eye to eye again.

    It doesn't seem to scale incredibly well. It's sluggish for me when I loaded up the planet python and planet humbug feeds into it via the 'import' facility. I'll see how it goes in the coming days.


    I used to have something kinda like this, but ultra minimalist and run by a combination of shell and python, called 'the feeder'. Unfortunately that was horribly broken, because it didn't use indexes, and used 1 file per feed entry. By the time I decommissioned it, the 1 hourly cronjob that generated the page would take 50 minutes to run. I never fixed it because the planet thing took off and made keeping up with things easier.

    In other news, I did nothing useful at the sprint, becuase threadedselectreactor defeated me, and my computer. Taking a gig of ram and a kill -9 every time you run it through trial.

    that gig of ram thing is an improvement though, before I patched it, it would hang within 30 tests because it had a queue that wasn't being cleared. I've no idea why yet, and I suck at threaded thinking enough to not understand TSR at all.
    Friday, October 7th, 2005
    2:56 pm
    buildslave braindump
    I was thinking today about the buildbot modifications I've been meaning to get around to for a while, and maybe this weekend during the twisted vsprint I'll be able to get something done on it.

    The idea goes like this. At the moment, buildbot works on the principal of having a single buildmaster, that many buildslaves connect to. An example is twisted's own buildbot. If you *clicky* there, you'll see a column titled 'OS-X'. That column is shiny, my wonderful powermac that IBM was so nice to provide me. Shiny is running a buildslave, which connects to the buildmaster that twisted runs. Every so often someone will make a CVS commit, which updates the source code. The Buildbot will then trigger a series of builds, one of those builds is the OS-X buildslave's builds, which look like:

    • Checkout Sourcecode
    • Compile
    • Cleanup
    • Test default reactor
    • cleanup
    • Test Threadedselect reactor

    This works great. tsreactor and defualtreactor get tested on a nice fast macintosh so that we know when someone's done something in a CVS commit that will break all twisted programs that run on that platform. The issue I want to look at is speed and robustness of the build process.

    If I turn off my machine at home, the OS-X column on the buildbot page dies until I turn my machine back on and restart the buildslaves (I haven't got it starting by default on boot yet - need to figure that out).

    The thing I want to try and achieve is make it possible for someone to type at a prompt
     foo@bar$ ./
    Creating buildbot user...
    Checking library availablity...Done
    Downloading latest buildbot source...Done
    Creating buildslaves...Done
    Installing startup scripts...Done
    so that it will be possible for someone who knows practically nothing about how buildbot works to install a buildslave. The other side of the coin is, I want the server to be able to cope with many slaves capable of fulfilling the same role, or better, figuring out which slaves are capable of what roles.

    There's factors involved: gtkreactor requires gtk to be installed, wxreactor requires wx, python2.3 slave requires to have python2.3 installed. So libraries will have to be tested for existance, and the library version numbers checked.

    Then there's the perks we can achieve. With 10 buildslaves instead of 1, we can run the different test batteries on different slaves. And only 2 in 10 machines have work to do, and the work is cut in half. If a machine goes down because that person goes on camp for a week (that was me two weeks ago) then practically nobody notices.

    So what I want to achieve is:
    multiple slaves per column on the buildmaster's web interface, and
    a script for installing a slave that requires practically no user intervention.
    Tuesday, September 27th, 2005
    10:25 am
    getting it right.
    Uche Ogbuji over at gets it right, when it comes to using xml and python. I've often referred to xml koolaid drinkers over to's little rants about how python and xml are completely unsuited to go together, now I have an article written from the java side of the fence that supports it :)

    Python is a clean, flexible language capable of expressing itself with minimum verbage. It needs XML like a fish needs a snorkel.
[ << Previous 20 ]