This is the approach I took to modernise Fairway (which is just one component of our suite). It doesn't free us from the limitations of the Extended Pascal compiler. It still works, but we want a way out.
Hello, I assume by your username that you're the author!
Transpiling is normally a little risky. Transpiling with a custom transpiler over ~500KLOC is downright terrifying.
I'd be concerned that the cure is worse than the disease.
I understand the allure of this solution. Bringing 30ish years of code into the modern era with a single piece of code sounds like a huge win. But you owe it to your employer (who seems really gracious and trusting) to ask:
1. Are there edge cases? Programmers are a clever bunch, and you can never be too sure about what features/bugs they've used/abused!
2. How will success of the transpile be determined? I'd use unit tests, but that'd take a lot of unit tests!
3. What % of that code path is critical? Can you get away with ensuring the functionality of a subset of the codebase? If so, could you just manually port that?
4. "Blaming the compiler" will be a reasonable excuse for any problem over the next year or two after deployment. I guarantee you'll be spending time proving that other peoples' mistakes are not your fault. How much flak are you and your boss going to take for this?
5. Since the code can be linked, might a more popular language (that will be known by a wider community) with a slow porting plan be reasonable?
6. Python is commonly used with scientific computing. Why does this case preclude it? If you are doing mostly numeric processing, that gets directly passed to the hardware anyway!
You have been given a rare chance to pay down technical debt. From what I understand of your solution, I believe the bespoke transpiler and semi-known language will ultimately increase it. And I know you love D, but if I were brought in as a consultant, I'd be recommending the .dll/.so solution with Python. If something is really time sensitive, use Python binding to work with other languages (avoiding premature optimization).
Even if myself or my fellow posters made a point that resonated, it might be too late for such discussions. I can't weigh the political costs for changing strategy now. So I'll wish you good luck, and hope that Extended Python doesn't have too many sharp edges for you to contend with. I hope to see a postmortem a few years from now detailing the outcome of this plan.
1. Unconsidered edge cases? We'll have to find out... We can always change the _source_ code to eliminate known unhandled edge cases.
2. We ourselves are heavy users of our software, and are likely to be the first to discover "oddities". More serious are subtle numeric differences, and we'll have to create tests against that. One approach to validating the transpiler is instrumenting the code. We could write a Pascal-to-Pascal transpiler that inserts extra code, for example logging the return value of every function. If we do the same for the Pascal-to-D transpiler, and do an identical run, we can compare the logs. And variations of that approach.
3. If a small percentage is unproportionally hard to translate mechanically, we can isolate that and translate manually.
4. I fail to see what you are getting at.
5. Of course we can link code. We think the community is wide enough for us.
6. You mean it gets passed to C.
Thanks for the suggestion. We have a working dual language situation with C++, we don't see that as an attractive option for our other tools. Why would Python be better than D?
I think point one is the concern of most people on this page. The most I've ever transpiled is 40kloc (this was for an R&D project, I'd think twice before doing it in production). ~500kloc is a different matter.
What % of the codebase are you familiar with? What % of the code is known by your team? If you had multiple, conflicting errors pop up at once, how much effort would it take to isolate them (not fix, just identify). What about in a section where no one has worked in a long time (assuming such a thing exists)?
Like any old codebase, I'm sure there are functions that have "just worked" for years (or in this case, decades). With a single action, every piece of code that's been proven solid over time is about to be suspect.
For point two, I think you have a lot more confidence in your ability to detect problems than I would in your shoes.
For point three, I'd much rather manually port necessary code piece by piece over a period of years. Unit testing like crazy while doing so. Thanks to linking, this is an option... One I'd recommend.
Point four is similar to majormajor's comment about the aftermath of the switch. I'm attempting to project beyond the immediate... This port could become a whipping boy... That could bleed into political fallout. No matter what technology you introduce, people will complain. Doubly so if problems arise. Adding a custom transpiler on top of that increases the risk significantly. The easiest transition path to a ${NEW_LANGUAGE} is through linking.
I'm not sure about Python vs. D. My instincts are towards the former, especially if I'm supporting SEM (STEM minus the T) grads as programmers. I also wouldn't easily write off interpreted languages (I'd rather work with them when possible).
I think you have your heart set on this path. But losing the binary safety net in favour of a bespoke transpiler is not a gamble I'd make with large amounts of production code...
Python has two big issues: performance and dynamic typing.
The solution of using a HLL and dropping down to C for performance strikes me as overly optimistic: the only language where one can legitimately claim that it's pleasant to use C for extra performance is Objective-C. Anything else will be different degrees of painful.
I didn't get the solution of using a dynamic lib and calling/extending that from Python. One would end up with 500kloc of Pascal and a Python one liner. All the work still happens in Pascal.
I think there is an understanding that the whole situation is less than ideal. I'm proposing a low risk method to transition from the current situation to a healthy one over the long term.
Having binary compatibility with the old version would be a great safety net while transitioning to another language. The main weakness of the author's plan, as the article explains it, is that they will be dealing with a lot of newly generated code at once (as well as an unknown number of unknowns). The plan would be fine if it was only ~10kloc, but ~500kloc is a very different beast!
Come to think about it... It would be wise to have a "Plan B" still written in D, but linked to the binary rather than transpiled. That way if anything goes wrong after deployment, users can temporarily use that rather than revert to Extended Pascal.
There probably isn't an easy solution for this problem. But maybe the author has found a silver bullet.
I brought up Python because it sounds like SARC is adjacent to scientific computing, and it is likely they'd be able to pick up physics/math/engineering majors that are already familiar with Python. Plus, I bet they'd be able to use some really good Python libraries (possibly replacing some of that 500kloc custom code with open source libraries).
The article makes it clear that naval architects (who are not expected to learn C++) are programming, so I doubt high performance computing is required in 100% of the code (at least I hope not). If there are genuine performance issues, tools alone won't save you, you need to hire programmers that know how to optimize (and they'll be fine writing C/C++ libraries with interfaces to Python).
I strongly suspect the justification that interpreted languages don't meet performance requirements is more ideological than empirical.
Besides, it's likely the existing binary will run faster than its transpiled descendant!
