2

We are using a Software as a Service platform that allows to create custom code which integrates in the platform and all its features (dialogues for common objects like Account, Customer, Address, and so on) and a GUI designer to create dialogues and reports. The platform uses a proprietary strongly typed, object-oriented programming language and a proprietary database.

We are facing multiple issues

  • the current implementation has bugs
  • the specification (requirements, expected behavior) may be incomplete and is distributed over around 1250 jira requirement issues (the former mentioned number 2500 does include tasks, bugs, tests, etc.)
  • the implementation (the code written) may already deviate from the requirement (incomplete implementation, misunderstanding) or is just buggy
  • the external team is new and took over the code from the previous external team
  • the platform has restrictions regarding mass data handling (creating 15 million rows (15 000 000) during an import proves to be challenging
  • budget for the external team size is tight
  • internal employees (apprentices) are available but are inexperienced
  • internal organizational politics are playing a role as well

Is this a duplicate?

I do not think so

Transpiler or rewrite?

  • My very basic understanding of ANTLR is that with the help of a grammar file code can be translated / transpiled from one language to another
  • But this does not solve that the translated code calls classes and methods for the proprietary framework that do not exist in the new target framework (maybe Asp.Net)
  • There is a possibliy inactive open source library that may convert some calls to the proprietary library to and outdated Asp.Net version

If we go the transpiler route we have to tackle

  • inexperienced with ANTLR and the lexer, parser, codegenerating process
  • rewriting the transpiled code to use the most recent classes of Asp.Net 8
  • creating test cases for the transpiled code to verify that it does behave as expected
  • deal with the design decions made (object names, object relations, which object / class takes care of what)
  • live with the database schema where some decisions could be based on the restriction the proprietary database has

If we rewrite from scratch (that one should never do and without losing sanity)

  • we could start creating test first
  • create a new database schema
  • take care of data migration after the new system can take over (big bang switch)

What are your thoughts and experiences?

Edit

Thanks for all the input. The comments and answers are immensly helpfull.

The question if transpiling the buggy code and fix it then might be still better than starting from scratch has been answered. The advise was that trying to transpile code requires a lot of Know-How and is only a minor part of the effort. It seems most advise against a rewrite but recommend to fix the current application.

The more important aspect than the technical issue is what the expectation of management is and what can be achieved given the current conditions (small external team and small budget).

nickw
  • 113
surfmuggle
  • 181

5 Answers5

15

Let's me take on your issues one-by-one:

the current implementation has bugs

Yep, and when you transpile such a code, what makes you think those bugs will not be transpiled as well? With a rewrite, your new system will also have bugs, but other ones, and not necessarily less.

the specification (requirements, expected behaviour) may be incomplete and is distributed over around 2500 jira issues (~ 1250 requirement issues)

Neither a transpilation nor a rewrite will fix that. Btw, when porting a program to a different platform, I would not use resolved issues from an issue tracker as a replacement for a spec, I would use the existing system and it's current behaviour as a replacement for a spec.

the implementation (the code written) may already deviate from the requirement (incomplete implementation, misunderstanding) or just buggy

A transpilation will not change this. A rewrite by people who take the old system as a template will probably copy these issues.

the external team is new and took over the code from the previous external team

You told us about an external team and and internal team, but forgot to mention who has the responsibility for maintaining what. However, I don't think any of your two approaches will change here much.

the platform has restrictions regarding mass data handling (creating 15 million rows (15 000 000) during an import proves to be challenging

Maybe going to a different platform might fix that. But usually solving an isolated issue like that in an isolated way is much cheaper.

budget for the external team size is tight

internal employees (apprentices) are available but are inexperienced

internal organisational politics are playing a role as well

These are all indicators for better not changing the platform, neither by a rewrite nor by transpilation.

What I miss completely in your post, however, is any kind of reason why you want to move away from the current platform. Are there hard technical or legal reasons why you need to leave the current platform? Or do you expect to make development cheaper or quicker on a new platform? How critical is the system for your business?

Hence, my best recommendation here is: first clarify what you want to achieve (and why), then start looking for solutions - and don't be fixated on "rewrite" vs. "transpilation". Think also about organizational solutions (like training unexperienced developers), or the option of keeping the old system as it is (and solve your major issues within the bounds of the platform).

surfmuggle
  • 181
Doc Brown
  • 218,378
6

Obviously you ask "rewrite or transpile?", but I'm not clear what the underlying issue is.

You mention the existing implementation is riddled with bugs. You mention that there is limited budget even to maintain the status quo. And you mention that you have nothing but apprentice labour available internally.

I'm afraid no special technique will solve the problem of a business that isn't living in the real world regards budget and staff skills.

Another thing that businesses are only slowly fathoming, is that virtually all kinds of technology has some kind of lock-in, and the point of SAAS (rather than periodic capital investment in permanent software licences) is to facilitate future price-gouging and private taxation of your business by the vendor once you are locked in.

Most people have forgotten when IBM used to rent the mainframe to you, and you weren't going anywhere else than IBM because you'd spent a decade with an army of staff writing the bespoke software for whatever IBM platform you were using.

That all said, work on your own bespoke transpiler will not pay off (I gather you're talking about making one, not just applying a COTS transpiler), and it's not obvious you even have the internal resources and know-how to perform either a transpile or a rewrite.

I would recommend avoiding do-something-itis, and just stick with what you can afford which is the status quo.

Steve
  • 12,325
  • 2
  • 19
  • 35
2

Why the whole hog?

You have listed numerous issues that make wholesale change risky:

  • Novice Staff
  • Tight Budgets
  • Third Party Supporters (misaligned goals)
  • Buggy/Weird behaviors
  • Lack of tests

From the sounds of it you also have a management that wants a well oiled machine but aren't willing to pay for one. You may want to investigate why they are unwilling. Is it actual financial constraints, were they burnt, is there a Hippo casting shade at you and your team?

Divide and Conquer

If you keep breaking the problem apart you will eventually find a problem so small that you can just solve it.

(Addendum: Except there are situations where you can't solve a problem in pieces - in which case this won't work look for something else.)

I can see several vectors where you can do this.

  • Write Specification Tests
  • Strangle off modules
  • Prototyping

Writing Specification Tests is probably the most obvious. Grab Gherkin and Cucumber. Write business language tests and write the steps to execute each statement against the system. Get your business stakeholders across how to write these, and how to read these. They will love them. Also teach your second/third tier support to use these to figure out customer issues. The more teams in the business write and contribute the better defined these system is. It also allows those business units to get clearer about explaining to you new functionality.

The best thing about these tests is that they are a visible win, a feedback mechanism that lets you talk to the business about what is wrong, what is needed, and how stable everything is. The first step on the road to money is getting the people that have it to understand what it is doing for them.

Another approach you could do is strangle off modules. Find a way to host the program and divert people from it to another program for particular activities. Start with a small area and slowly move functionality across. I can't speak for your situation but there are many ways to do this. The win here is that you can demonstrate greater reliability in the new modules, they can have the testing from day dot, and be in a platform that is useful.

Strangling off modules is itself a win that can be used to demonstrate to the business where their money is going, and what effect it is having. You can also show case what differences the testing/platform/language have, and also demonstrate staff training.

Prototyping is closest to what you have in mind. Throw up as quickly as possible a replacement for the entire system. Throw it up in the platform/language of choice. Hit the most important functionality. The point is not to replace the current system. Its to demonstrate the technology, development patterns (like unit tests/pipeline/etc), and train the developers. The whole point of the prototype is to be deleted after demonstration.

What it achieves is a counter example that is usable enough that the business can decide if its better. They can see the costs as they can extrapolate from the prototype, and they will see what it gets them. The best part is that it can be payed off as a team collaboration/training exercise.

None of the above solve the problem with the current system but they work to increase the social capital with the business and to alleviate one of the core problem you currently have (Lack of specification, tight budgets, Team Experience).

Kain0_0
  • 16,561
1

Transpiler or rewrite?

Just convert it.

Easily 80% of my entire career has been doing this. You get one requirement: make it do what it did before on this new system.

It isn't a rewrite and you don't have to use a transpiler (these can cause more problems then they solve). All you need is coders who are familiar enough with both languages and systems to convert from one to the other.

We are facing multiple issues

Every single issue you mentioned is an issue even if you don't covert it. Thinking those are impediments to converting it just shows you have the wrong attitude about converting it.

Converting software doesn't make it perfect. But it is absolutely the most thorough form of code review. If you do this, even if you don't use the new version, your knowledge of the code will be better for it.

Be very reluctant to make any improvements to how this works. Your goal is the conversion. You will notice things that need fixing. Don't fix them yet. Convert first.

If you were hoping things would be better with the converted code, if that was your main reason for doing this, forget it. Converting wont make the mess better. It just moves it into a new stack.

we could start creating test first

This is an option whenever you write code. I've done it in shops that won't support unit tests. I just keep my tests to myself. If you want to do this there is never a reason not to.

create a new database schema

Dear god why?

take care of data migration after the new system can take over (big bang switch)

Well, if you insist on changing the schema in the middle of this (seriously why?) make sure you automate the migration. Test the hell out of it before you take it live. Have a plan for when this fails.

candied_orange
  • 119,268
0

My Experience is that "lock in" is a real thing and that this problem is unsolvable. It's the "should I rewrite or refactor" problem, which is already a no win situation, but refactoring is even harder than normal and a bunch of extra costs and contracts have been thrown in to rewriting.

My thought is, you may have found a problem that AI is A solution for. "AI" by which I mean the current Chat GPT style offerings, seems to be passably good at problems like "explain this code" or "write this in language X"

I wouldn't expect 100% automation, but it might take away the "configure ANTLR" problems and leave you with just the "finish and test this code" problems. It might be worth a try.

The normal solution I would suggest is not "rewrite" but "write v2". Write a new system which isn't a direct replacement, but competes with the old to solve the same problem.

This allows you to pick a smaller scope or problem to work on, address fundamental problems or changes in problem statement and business case that might have occured over the years and crucially move forward as a business while working on this issue.

Another solution that just occurred to me is that you could suggest migrating to a different SAAS product/supplier. SAAS companies usually promise that they can migrate in existing processes and offer consultant help to do the work.

You might be able to get a good deal and reduce costs by switching, as the new company will be incentivised to aquire you as a customer. Indeed even the threat of moving might elicit more help from your current supplier.

Ewan
  • 83,178