Incremental Computation Model Refresher

I originally planned to describe the improved dependency tracking algorithm for the Rust’s incremental compilation as the follow-up to the last post. However, while doing a draft of that, I kept getting confused about cached results and their exact relationship to nodes in the dependency graph. I think it’s useful to write down what my mental model for incremental computation in the compiler is, especially since it’s slightly different than what’s described in the incr. comp. alpha version blog post and also in the incr. comp. RFC.

Better Dependency Tracking for the Rust Compiler

The Rust compiler uses dependency tracking in order to find out what needs to be re-compiled during incremental compilation. The model of data dependency currently used by the compiler is less powerful than it could be though. In this blog post I’ll show what its shortcomings are and what a more expressive model could look like.

Attempting to Use Rust’s Type System for Statically Checked Dependency Tracking

In order to support incremental compilation, the Rust compiler tracks dependencies between the pieces of data it reads and produces. It does this so it can determine what needs to be re-computed when something in the source code changes. I have long suspected that Rust’s type system would make it possible to ensure, at compile-time, that the dependency tracking system is used correctly, that is, that the people writing compiler code don’t forget to register data dependencies in the appropriate places. After all, the Rust’s type system, initially designed for safe memory management, has proven to be useful in other areas too, like preventing data races and statically checked fork-join parallelism. In the following I’ll describe how I tried to achieve similar correctness guarantees for the dependency tracking system via rather simple and ergonomic means. Did I succeed? Read on!

rust-gdb. rust-lldb.

Unbeknownst to most Rustaceans, two little helpers have found their way into almost any Rust installation over the last few months: rust-gdb and rust-lldb. These are two small wrapper scripts that will start the underlying debugger with Rust pretty printers enabled. The pretty printers are Python-based extensions that hook into the debuggers output rendering, hopefully making it easier to inspect your Rust program’s runtime state. This post will describe what rust-gdb and rust-lldb can do for you, what their limits are, and how to use them (hint: it’s easy!)

Rust Debuginfo and Unique Type Identifiers

Over the last few days I’ve been working on some code that creates unique type identifiers for the types in your Rust programs. For scenarios where the intermediate LLVM code of multiple crates is merged into one compilation unit (as is the case when link-time optimization is enabled), LLVM needs a way of telling which type debuginfo is the same in both crates. This allows it to get rid of duplicate data. Also, when there is a conflict in the type identifiers (i.e. two different types have the same identifier) LLVM will abort with an assertion, an issue that has been poping up a few times lately. So I set out to create something that would solve this issue once and for all, a task that turned out to be more difficult, but also more interesting than I anticipated.

Mozilla Contract

I’m happy to tell you that I’ll be working on Rust’s debuginfo support in the context of a contract with Mozilla. I have been to able fix some bugs and add a few features after my GSoC project. However, having the contract means that I’ll be able to consistently spend time on Rust―two days per week over the coming months, to be precise. Which is a fabulous thing, needless to say :)

Apart from fixing existing and emerging bugs, there’ll be two major areas that I’ll work on:

What you call the present, we call the past

Now that the Summer of Code is over, it seems like the proper time to give an overview on the state of debugging information in Rust and to talk a bit about what I was able to accomplish this last three months. In order to make this an interesting read for you, I thought it might be a good idea to combine this with a guide on how to use the new features in your projects.

#Generics and Debug Information in Rust A couple of weeks ago I added support for generics to rustc’s debug info generation and found that it would be an interesting topic to write about. So, what are generics? In short, they allow program items―such as functions, structs, or traits―to be parameterized with type variables. That is, expressions within those items―such as locals, arguments, or fields―are declared not with a concrete type, such as int, but with a not-yet-concrete type designated by some name:

// T1 and T2 are type variables
struct Pair<T1, T2> {
    a: T1,
    b: T2
}

In order to create a value of a generic type such as the above, concrete types have to be substituted for the type variables. This is a very useful tool of abstraction, allowing to have just one List<T> instead of IntLists and FloatLists and TransmogrifierLists, to name one very common example. It is so useful that the concept of type variables can be found in nearly any statically typed mainstream and not-so-mainstream language developed not too long ago. Quite the success story, given that “generics is for academics only” :P

Iterator Blocks for Rust - Feature Survey

In the first concrete installment of my series on iterator blocks I want to explore the possible feature set that could/should be supported in Rust. In the following paragraphs I will try to collect things that I think one would expect from iterator blocks in Rust. Mind, however, that this is just my opinion so far. I'd very much like to discuss this with anyone interested on the mailing list. Nevertheless, I think a collection like this is valuable as a point of departure—even if most of it won't make into a hypothetical final implementation.

I categorized the points below into must haves, nice to haves, and things to think about. The syntax I use here is very close to C# and should not be considered a proposal for the real syntax. This topic is big enough to warrant its own post.

Visibility Scope in Rust Debug Info

The last few days I’ve mostly been working on the creation of proper scoping information in Rust debug symbols. As often is the case, it soon turned out that this is a deeper, more complex topic than it looked on first sight. This post will provide a small chronology of my journey into its unexpected crevices.

</img>

#Iterator Blocks for Rust

Lately, I’ve been thinking about what it would take to implement C#-inspired iterator blocks for Rust―naturally something that has been discussed before―and decided to try to bring some order into my thought process by means of this blog.

</img>

Iterator blocks (in C# parlance) or generators (Python) are what is commonly associated with a yield statement or derivations thereof. The idea behind them is to allow the programmer to write something that looks like code typically found in the context of internal iteration, but then automatically transform this code into something that has the properties of an external iterator.

#Weekly Status Report #5

Not many words from me this time. This week I worked on:

• Getting the big Work Package 4 pull request ready for merging and extending it with some fixes (e.g. supporting the new memory layout of unique vecs).
• Added support for debug info about locals declared with destructuring patterns. This works very stably already.
• Fixed function argument handling which has been broken for a while now. Unfortunately, there is still an issue with function argument destructuring. I don’t know really what’s going on here because the code should essentially be the same as with local variables. This will be the first thing to look into next week.
• Spent some spare time doing general refactoring and naming cleanup work on syntax::ast and rustc::middle::trans.

#Weekly Status Report #4

tl;dr ― Wrapping up debug info generation for enums and some venturing into the LLVM code base.

#Weekly Status Report #3

tl;dr ― Enum support is in the works.

A rather short update this week. I’ve been working on supporting the various kinds of enums, starting with c-style ones (that is, their values just consist of a discriminant and have no fields). This worked out rather nicely, as they have a direct equivalent in C and LLVM’s DIBuilder, DWARF, and gdb have good support for handling them.

#Weekly Status Report #2

tl;dr ― Memory layout debug info finally done right :)

Another week past, another report due. Last week I already started thinking about how to handle data structure memory layout in a less fragile way than the compiler does to date. Some of these thoughts can also be read about in an email I posted on the mailing list on Monday. The short version is this:

#Weekly Status Report #1

tl;dr ― It’s been great. Wrote some documentation, wrote quite a few test cases, found and fixed a bug regarding struct and tuple padding.

#Summer of Code 2013 w/ Mozilla and Rust You probably began to feel it too last week, the changing of little things: The food tasting delicious, an unexpected smile on your lips, beautiful people everywhere you look. The tune of the world finally spinning into harmony, if you will. And the spring in your step it has not subsided since. Maybe you don’t know where it is coming from, this assurance that this summer is going to be good, to be great even—so let me tell you.