By: Nick Hodges
Abstract: This article provides an overview on the status and future of the Delphi compiler.
Introduction
This article is intended to give all you Delphi users some insights into what we are working on and considering for the Delphi compiler. We here at the Embarcadero World-wide Headquarters know all too well how you all love your Delphi compiler, and since there are some cool language and compiler technologies brewing in the R&D labs, and we wanted to make sure you were up to date on the future of your favorite compiler.
Some History
But first, let me give you a little history about the compiler. Most of this is probably not news to you all, but it is important to remember it. First, the Delphi compiler has been around for quite a while. It has proven itself time and again as a fast, modern tool, and it has provided you folks with lots of cool and modern features like generics, anonymous methods, inlining, and more. However, because of its rich heritage, there actually is still some code in it from the old Turbo Pascal days, and those days were, well, a long time ago in a galaxy far, far away. There are still language features and code constructs hanging around that many of you probably didn’t even know existed. For instance, how many of you know that you can still create comments with (*...*)? Or how many of you know that this still compiles:
var
WeirdLookingArray:array(.1..10.)ofstring;
Come on, be honest. Not too many of you knew that.
At any rate, the compiler maintains these constructs, and many others, for backwards compatibility. Backwards compatibility is really, really important to you and it is really, really important to us. Let me repeat that: Backwards compatibility is really, really important to you and it is really, really important to us. (Hopefully the italics help emphasize the point. Okay, now we can move on.) However, many of these features are not used by a large portion of you anymore. Nevertheless, we still maintain them, test them, and certify them for each release of the compiler. That’s a lot of work. But since you all are probably not using all the stuff that is in there. (Come on, don’t even try to tell me you knew about that “parentheses and periods” thing....), it ends up being work that most of you don’t care about. Supporting less used language features costs time that we could spend working on new capabilities.
The above examples are just that, examples. But many of these code semantics are features that actually can prevent us from making changes and modernizing the language. Let me be clear before I go any further: I’m not going to be talking about eliminating features; I’m merely talking about an issue that the language currently has. We have no plans to remove any language features or break your code. Again: we have no plans to do that. We do have some interesting new plans, though.
Therefore....
Therefore, we’ve been thinking long and hard about this. Specifically: How can we move our compiler forward to enable it to keep doing all the things that modern, powerful, compilers do? How do we move the compiler forward adding cool new stuff while at the same time preserving your existing code?
Well, the answer to that question is neither short nor easy. It also isn’t something that we take lightly. We could just rewrite the whole compiler front end to create a new Delphi language that does what we want and which then abandons a lot of the code out there that you all have. But that would be bad. Really bad. We definitely don’t want to do that. Let me repeat that: we have no plans, no intentions, and no reason to make your existing code no longer work.
What we do want to do, though, is to be able to continue to move Delphi forward in order to keep it at the cutting edge of native code development (and shoot, general development, for that matter). But sometimes, moving forward is hard because of things that are hanging around from the past. Delphi is a very type-safe language, but not completely type-safe. You can still do things like call GetMem or create an unbounded array (array[0..0] of integer) and do all kinds of crazy things with raw memory using such a reference. That’s a powerful thing to be able to do, but it also means that you can be a lot less descriptive about what you are doing, and so the compiler lets you go on your merry way. In many cases, this may be exactly what you want, but this is a primary example of opening things up so that you can “shoot yourself in the foot”. Delphi lets you ride without a seatbelt, but the fact that you can do that means that there are things Delphi itself cannot do. Wouldn’t it be nice if you could still drive your cool car, but with a nice three-point harness and a full set of airbags. So what’s a compiler to do, then?
The Compiler Front End
Compilers generally have what is commonly called a “front end”. The front end take code, parses it, and creates a tree structure that represents the semantic intent of the program. It’s the part that understands the language syntax and ultimately defines what the language is. In Delphi, the front end knows how to read units, for loops, begin...end pairs, variable declarations, generic classes, all that “Delphi Stuff”. It is also the front end that creates the DCU files that currently are compiler-specific. It’s the part, too, that deals with all the historical syntax above. So the question becomes: How to modify the Delphi front end to enable new and modern language features while still ensuring that your valuable code works?
What if we created a new compiler front end that gives you the choice between a new, non-legacy syntax and the older way of coding? (Let me be clear again: we are not talking about removing language features. If you are firing up the comments to ask “So, what code constructs are you going to deprecate?,” I answer that now by saying “None. All you code will still work.” We should be pretty clear on that by now, right?) Or better yet, how about a new way of viewing existing code and “new” code so that they can work together, allowing you to put new code in new modules while keeping your old code working as it always has? Existing code would all still be able to be injected into the new way of doing things, but if you wanted to use the “new and improved” stuff, you’d have to do it in the new type of code module? And how about if, while we are at it, we end up creating a DCU (or something akin to what DCUs are today) format that aren’t compiler version specific. How does all that sound?
That sounds pretty cool to me, but of course, doing something like that isn’t trivial, and would require a serious amount of time and effort. But something like that would be worth it, no?
But the front end is only half the battle. There is also the compiler’s back end.
The Compiler Back End
For you non-compiler mavens, the “back end” end of a compiler is the part that reads what the front end produces and actually writes out the machine code to produce usable binaries or something that is useful to some type of runtime somewhere. The back ends of our Delphi and C++ compilers currently produce 32-bit binaries for Windows. But of course, back ends don’t have to be limited to doing 32-bits and Windows, now, do they.
One of the new things we want the back end to do is produce 64-bit binaries. And so producing 64-bit binaries will require a new back end. We could just take our existing 32-bit back end and hack around it to be a 64-bit back end and toss that over the fence to you guys. But that wouldn’t be the right way to do it, especially considering that our C++ and Delphi compilers currently have different back ends. Why do the work twice when you can, well, do it once? And of course, a 64-bit project is far more than the compiler. We want to make sure that the debugger, the editor, the file system, refactoring, modeling, documentation – everything – is done right and totally ready to be able to let you develop 64-bit applications with no limitations. That will take some time.
Instead, the right way to do it would be to write a new back end. And if you are going to write a new back end, the right thing to do would be to write a single back end that can be used for both Delphi and C++Builder. And if you are going to do that, you’d want it to be a back end that is architected in such a way so that it could be a bit more flexible with respect to what architecture it actually targets. Interesting thought, eh?
So it makes sense to do a whole new, unified back end that will work for both Delphi and C++Builder. But doing that takes time. More time than we’d like, frankly, but these things cannot be rushed. As a result, doing it right, making it work the way it should, and creating a proper compiler architecture to benefit both Delphi and C++Builder will be a non-trivial endeavor. Doing it right is important to us, and I suspect it is important to you as well.
Interestingly, the time needed to create this new back end and build a 64-bit product creates a window of opportunity for us to do the new Delphi front end. So, we’ve made the decision to take the time to do all the backend work needed, allowing us to do the interesting front end stuff as well. We aren’t going to hack something together and throw it over the fence. We want to create a new compiler that will be really cool. We assume you do, too.
The Delphi compiler is at an inflection point. And it seems like this inflection point affords us (and, ultimately, you) the opportunity to make that inflection point really mean something.
Bottom Line
What, then, is the bottom line here? Well, to get right to the point (finally!) – with some major compiler work being done on the new backend and the 64-bit compiler, this seems like a good time to “make a move” and transition to this new compiler architecture. We expect the 64-bit version of Delphi to be ready in mid-2010. Because the compiler is the first step in a complete 64-bit product, we are planning to release a preview of the 64-bit compiler in mid-2009 so that those of you who are extremely eager to see a 64-bit Delphi can get started early.
We know this is a change, but we believe – and we hope you agree – that it is worth taking the time to get it right and to make the very cool changes that will keep Delphi on the cutting edge language technology. Once we take the time to build the new compiler, there should be a lot of cool and new stuff we can add from there. And you guys love cool and new stuff, don’t you?
Conclusion
Delphi has to move forward. C++Builder has to move forward. Our compiler has to move forward. To move forward, the compilers have to embrace new sets of features that modern developers demand. Both languages have to provide the solutions that you are looking for now and in the future. In order to produce these new features we are going to have to, as the saying goes, break a few eggs to make an omelet.
But you guys have about ten bazillion lines of code out there that you want to continue to work. We totally get that. So, we are working to create a “new Delphi” and a new compiler architecture, to keep your existing code working, to emit 64-bit binaries using both Delphi and C++Builder, and maybe a few other kind of binaries while we are at it. And it all has to be done right so that it all works for you.
And you know what? That is exactly what we plan to do.
