Answering questions by looking at code

A contributor recently asked a question in #introduction that I couldn’t answer immediately, but I suspected that I could find the answer without too much work. I also decided to write down the steps I took and the reasons I took them, with the intent of creating a worked example of finding answers in the Firefox code.

The question: how do I determine if a given URI pointing to an RSS feed is subscribed as a live bookmark?

My first instinct when looking for code I have never touched before is to start with the visible strings in the Firefox interface. In this case, I went to slashdot.org and clicked on the RSS feed button to take me to the RSS feed viewer. Under the options for possible subscriptions, there is the “Live Bookmarks” choice. I’m going to find the code that handles subscribing a live bookmark, which should lead me to the way to find out if a feed is currently subscribed.

So, off to MXR: http://mxr.mozilla.org/mozilla-central/search?string=Live+Bookmarks&find=&findi=&filter=^[^\0]*%24&hitlimit=&tree=mozilla-central

Strings are usually stored in special localized files, where the actual string is referred to by a unique name. In this case, we’re looking at subscribe.properties, which shows us that some code should be using the name liveBookmarks to show the localized string in the feed viewer window.

Let’s look that up: http://mxr.mozilla.org/mozilla-central/search?string=livebookmarks&find=&findi=&filter=^[^\0]*%24&hitlimit=&tree=mozilla-central

Out of the results, subscribe.xml sounds like something I want to read: http://mxr.mozilla.org/mozilla-central/source/browser/components/feeds/content/subscribe.xml#53

Unfortunately, that line turns out to be not very interesting. However, let’s see what we can find for liveBookmarksMenuItem, which sounds like it might be the item for the live bookmarks option in the dropdown: http://mxr.mozilla.org/mozilla-central/ident?i=liveBookmarksMenuItem

Let’s look at FeedWriter.js from the results: http://mxr.mozilla.org/mozilla-central/source/browser/components/feeds/src/FeedWriter.js

Woah, that’s a lot of code! Let’s take a step back – we care about the subscribing action right now. Going back to subscribe.xml (http://mxr.mozilla.org/mozilla-central/source/browser/components/feeds/content/subscribe.xml), notice that the button is called subscribeButton. That sounds really useful! Let’s look in feedwriter.js again for that (just a regular ctr+f operation): http://mxr.mozilla.org/mozilla-central/source/browser/components/feeds/src/FeedWriter.js#859

Now, let’s find the function subscribe (ctr+f for subscribe:, as that’s commonly how member functions are defined in the Firefox source): http://mxr.mozilla.org/mozilla-central/source/browser/components/feeds/src/FeedWriter.js#1241

Reading the source of subscribe(), we come to
http://mxr.mozilla.org/mozilla-central/ident?i=addToClientReader

We can see the definition of addToClientReader at http://mxr.mozilla.org/mozilla-central/source/browser/components/feeds/src/FeedConverter.js#390. Let’s look at what else is defined here – this seems to be implementing the nsIFeedResultService interface, so we can look up the nsIFeedResultService.idl file in MXR: http://mxr.mozilla.org/mozilla-central/source/browser/components/feeds/public/nsIFeedResultService.idl.

Unfortunately, the service seems to be dealing with actual feed contents, and it doesn’t look like there’s a direct function to help us – let’s step back and see how addToClientReader is implemented.

http://mxr.mozilla.org/mozilla-central/source/browser/components/feeds/src/FeedConverter.js#448 is an important line – let’s look at addLiveBookmark (I just clicked the name again): http://mxr.mozilla.org/mozilla-central/ident?i=addLiveBookmark

Take a look at http://mxr.mozilla.org/mozilla-central/source/browser/base/content/browser-places.js#457. This shows us how a bookmark dialog is created, but not the actual subscription process. However, the important part is line 471, which shows us the type "livemark". Let’s search for that string and see what other code uses it: http://mxr.mozilla.org/mozilla-central/search?string=livemark

Woah! There are a lot of results! Since I’m not certain what I’m looking for at this point, I find the best thing to do here is to scan all the results quickly, looking for interesting snippets or filenames before opening any particular file. Partway down, there’s a file called test_bug636917_isLivemark.js (http://mxr.mozilla.org/mozilla-central/source/toolkit/components/places/PlacesUtils.jsm

Searching for livemarks in this file, we find line 2234 which defines the livemarks object as the nsILivemarkService service. Let’s look that up!

First the idl definition, to see if this service will be useful: http://mxr.mozilla.org/mozilla-central/source/toolkit/components/places/nsILivemarkService.idl. There’s a method called getLivemarkIdForFeedURI (http://mxr.mozilla.org/mozilla-central/source/toolkit/components/places/nsILivemarkService.idl#107), which looks to be exactly what we want – it gives us a -1 result for a non-registered feed and another number for a registered one.

So, to sum up – we have found a service that will take a URI and return a value that can be used to determine whether a given feed is a subscribed livemark, and we have learned that livemarks are the internal name for Live Bookmarks. The only bit remaining now is to get access to this service, but we can either use PlacesUtils.livemarks for that (if we import PlacesUtils.jsm), or just use a normal getService call. These are just details; the hard part of answering the original question is complete!

A Firefox regression hunter VM!

I am happy to announce the first public unveiling of my Fox in a Box project. The enthusiasm for my idea of putting together a dev VM was loud and clear in my last post, mainly from testers who expressed interest in hunting regressions at a more granular level. The README in the first link hopefully contains enough details for anybody to get themselves set up with a minimal Ubuntu 11.04 VM with a preset clone of mozilla-central ready to build. I experienced clobber build times of 30-45 minutes in my tests; your mileage may vary. So please, run $ git clone git://github.com/jdm/foxinabox.git, follow the README and let me know how it goes for you!

A Firefox dev VM?

I’ve been toying with the idea of creating a virtual machine that’s ready to build mozilla-central out of the box. The original idea I had was that it would be really easy to get new contributors started, and it would allow us to set up finicky things like the hgrc correctly. However, it was pointed out to me that build times in VMs are pretty terrible, which does make it less attractive. However however, I have seen comments from technical people in bugs that they would like to help find regression ranges smaller than 24 hours, but they lack the time to figure out how to set up a build. Is it worth creating a VM for finding regression ranges? I can imagine it being really pleasant to start a VM, run a script, and check back in every 30 minutes to see if the next build is ready for testing. Alternatively, implementing Kyle Huey’s proposed solution would also be a pretty big step to easing those pains.

A spoonful of sugar helps the .mozconfig go down

In my work with new Mozilla contributors, I’ve seen a number of people get hung up on the whole .mozconfig voodoo madness. Yes, we made it possible to download the source and build it without having to even think about creating one, but as soon as you want to use a debug build, or build Fennec, the dream is over. Therefore, I’ve put together mozconfig.py which is designed to walk users through the process of creating a working .mozconfig file tailored to their needs.

The best way to use it at present is as follows:

$ ./mozconfig.py 2>~/mozilla-central/.mozconfig

This will send the generated file contents directly into the relevant .mozconfig, so all that’s left is running make. I’d love to be able to programmatically determine whether the program is actually invoked this way, so it could give helpful instructions otherwise. However, I’m open for ways in which to improve it (keep in mind the target audience of “people who need to work on common tasks that require changing .mozconfig options”), and I’m really keen on ideas of how best to market this (wiki pages, etc.) so that it reaches the widest possible audience.

Making bugs more attractive for other people to fix

There are two takeaway points that I would like to state up front:

  1. when you find a bug that’s well-scoped and you have a fairly clear idea of how to fix it, add [mentor=yournickname] to the whiteboard.
  2. when stating that a bug is good for someone else to work on, please please add a comment with relevant MXR links and information on how to fix it.

I’m absolutely thrilled to see people marking bugs as being good for other people to fix. That’s what the “good first bug” and “mentor=” annotations are all about. It’s a message from someone who understands the issue, and it indicates that the bug is fixable with a reasonable amount of effort on the part of someone who is not a core contributor.

I’m going to go out on a limb and say that using [good first bug] is no longer desirable. The list of bugs with this annotation has grown beyond a manageable size (not in a good way), and it can appear like an impenetrable barrier for someone who shows up with the idea that they “want to work on Firefox.” Now, I subscribe to the RSS feed of this list, so I see every new addition and will likely to ping the culprit on IRC and ask whether the bug in question would make a better mentored bug. What is a mentored bug, you might ask? Segue!

Mentored bugs are the new hotness. A mentored bug is one in which someone with knowledge of the code in question has added the [mentor=name] annotation to the bug’s whiteboard, and represents a contract that:

  • the issue is fixable
  • a fix is desired
  • there exists a plan for how to proceed with fixing it
  • the specified mentor understands the problem and proposed solution

These are the absolute basics. Without these guarantees, a bug should not be assigned a mentor, and certainly shouldn’t be proposed to a contributor looking for work.

Furthermore, I propose that bug mentorship be a social contract as well. When I was a volunteer contributor with a non-Mozilla day job, I found it difficult to get on IRC during work hours, so I wasn’t able to have real-time conversations with my mentor. Moreover, the work I was doing was understood by very few people, so the folks on IRC in the evenings couldn’t really help me either. Therefore, I recommend that bug mentors make the commitment to respond to emails from contributors regarding the bug in question, because that can significantly ease the burden for eager volunteers in situations like my own. I have carried on several long email conversations with people working on my mentored bugs, consisting of advice, links to documentation, and technical support, and they have resulted in patches for those bugs every single time.

So, should you become a bug mentor? Absolutely! How can you decide whether a given bug is a good candidate for mentorship? Here’s a helpful checklist that I go through when considering that question:

  • Does it fulfill the basic requirements above?
  • Am I willing/able to answer questions about the proposed solution, or do I know who to point a contributor towards if I don’t have the answers?

That’s it! Bug mentorship is easiest when you have a complete understanding of a problem, but all that is truly necessary is enough knowledge to act as a willing guide for someone who knows less than you.

The final point that I want to emphasize – when you’re marking a bug as being good for another person to work on, please take a few moments to dump your investigative work in the bug. Function names, MXR links, steps to reproduce, explanations of the problem and how it can be solved: these are the stepping stones that allow a contributor to start looking into how to solve the problem without waiting to get in contact with you. The less latency involved, the better. Remember, these bugs are often being approached by people with very little experience in the code in question – spell out the assumptions you make, and you’ll make someone else’s job easier.

Appendix A – Types of bugs that are good candidates for mentoring:

  • Reliable edge-case crashes – easily reproducible, typically not in the critical path, great visibility (contributor can demonstrate behaviour change to third party)
  • Bugs containing simple (or at least explicitly spelled-out), verified testcases – easily reproducible, good visibility
  • Well-scoped tasks (ie. change all uses of X to Y, remove Z) – easily explained

Two years he walks the earth

Two years ago today I sent an email to Benjamin Smedberg saying that I wanted to help out with Electrolysis. I regret nothing except how quickly the intervening time has passed. Bring on two more years!

The Fast and the Furious: bzexport

If you’re a bzexport user, you’ll want to pull the latest revision. It’s recently been getting slower due to the number of http requests that need to be made (the tradeoff here is that it’s also been getting more correct), but I’ve pushed three patches that have reversed that downward slide into the molasses. By sending more data than is strictly necessary, caching information such as user name to email translations, and avoiding fetching data that rarely changes, I’ve noticeably improved the running time of bzexport on my machine. Give it a shot!

If you’re not a bzexport user, and you write patches that are submitted to bugzilla, you’re missing out. Bzexport simplifies that process of getting a patch out of your local machine and into the hands of your reviewer, making you a happier and more productive person.

nsCOMPtr has never been so pretty

Jim Blandy announced his archer-mozilla pretty-printers for Spidermonkey late last year. I’ve used them a few times while working on some JS proxy bugs, and I’ve found them to be invaluable. So invaluable, in fact, that I’ve written a bunch of pretty-printers for some pain points outside of js/. If this prospect excites you so much that you can’t be bothered to read the remaining examples, you can acquire the code from its hg repo. Please note, you’ll need to be using gdb >7.2 (meaning probably trunk at the time of writing) due to dependencies on the dynamic_type feature which was recently added.

What’s supported right now:

nsCOMPtr<nsIFoo>

If you print an nsCOMPtr, you’ll see one of two things:
[(nsIFoo*) 0x0]
or
[(nsConcreteFoo*) 0x12345678]

In addition, pretty-printers chain, so if there’s a pretty printer for nsConcreteFoo*, you’ll see that printed instead of just the pointer value. Furthermore, nsRefPtr and nsAutoPtr are also supported, and maybe nsWeakPtr (untested).

nsCString/nsString

It prints the string inline, up to the limit specified by gdb. This probably only works for ascii strings at the moment; the code is a bit of a mess.

nsTArray/InfallibleTArray

You’ll get output along the lines of [0x12345678, 0x12345678] at the moment. Like the smart pointers, each printed element will be pretty-printed if possible.

nsStandardURL*

You’ll see the spec printed. This works wonderfully when you’re looking at an object with nsCOMPtr<nsIURI> foo, where foo is actually pointing to an nsStandardURL object.

Tagged reference counts

You know in cycle-collected objects when the refcount shows you mTagged = 0x8? Yeah, not any more.

PRPackedBool

I found the existing display of mBoolVal = 1 '\001' to be silly.

What’s coming up?

I’ve got a list of things that still need fixing/improving, and I’m open to suggestions/patches.

  • Auto arrays – the kinda-sorta work right now, but it looks gross.
  • COM/void/observer arrays – don’t work at all
  • Hashtables – don’t work at all right now
  • Stricter regexes for matching types – things like pointers to arrays are matched by the nsTArray printer right now
  • Auto strings – I don’t remember if they don’t work at all or semi-work
  • PRUnichar* – doesn’t work.

and so on and so forth. It’s really not difficult to write more printers, and it improves the output of gdb immensely in my opinion. Give it a shot! The instructions for how to set this up can be found in the README file. You’ll need a version of gdb > 7.2 (probably trunk, at the time of writing), since my printers rely on gdb.Type.dynamic_type.

Self-serve tools: now more likely to work

If you’ve given my self-serve tools a try (in particular, cancel.py) and had it claim that it couldn’t authorize you, it’s time to give it another shot. Steven Fink, being the wonderful person he is, dove in and found some weirdness going on with my usage of urllib and the self-serve API performing redirections. The end result is that the tool is significantly more likely to work for you, and if it doesn’t, I am really interested in seeing the output of python cancel.py -d, which makes urllib be significantly more verbose about what it’s sending and receiving. Good hunting!

Build smarter, not harder

I spent the past six weeks roaming around Europe with a netbook, and used that time productively to get some work done on Firefox. Part of that involved building on Windows for the first time, and experiencing the joy of pymake. However, I found the extra characters required to fire off incremental builds with pymake pushed me just past the pain point required to get me to write some sort of automation. With that in mind, I introduce smartmake, a tool to allow you to specify as little information as possible to build incrementally while still ending up with a working finished build.

Here’s how it works: I’ve encoded some basic dependencies into a python file (any changes to layout/build, netwerk/build, js/src, etc. require a rebuild of toolkit/library, any changes to layout/ or dom/ require rebuilding layout/build, etc). You pass a list of srcdir directories that have been changed to the script, and it prints out a list of of build commands joined by &&, ie:

$ smartmake ipc/ipdl dom/src/geolocation
make -C objdir/ipc/ipdl && make -C objdir/dom/src/geolocation && make -C objdir/layout/build && make -C objdir/toolkit/library

It’s a pretty dumb tool at the moment, and there are certainly lots of edge cases that don’t actually work correctly. However, I found it useful enough in Europe that when I returned home to a different machine, I missed it. It’s hardcoded for my own setup right now, but I’ll try to make it more general if people are interested (ie. the objdir and cmd variables could be read from a config file). Hit me up with any requests or if you have wonderful ideas for how to improve it.

Update: I’ve genericized it a small amount. Now, you’ll need to create .hg/.smartmake like so:
[smartmake]
objdir: objdir/
cmd: make -C

before the updated smartmake will allow you to continue. Notice that this is per tree, not global. Furthermore, smartmake.py is designed to be used as a tool that outputs a command line – I have a shell alias that pipes its output to sh.