This turned out to be longer than I had planned (it had started as a comment), but I hope the added length/details are helpful and you find them justified.
Agile is not specific to CRUD apps
Most of the literature on agile seems to be biased towards CRUD type business applications where the user is pretty much aware of what is going on behind the scenes. (That's fine because most of the code being written probably belongs to this class.)
I think this is because it's easier to create easy-to-follow examples of this type, not really because the methodology is aimed at those kinds of systems. If you create a not-so-easy-to-follow example, you risk getting the reader stuck trying to understand the example when your point was supposed to be teaching the reader about agile concepts.
User Stories != Requirements
For this type of application the relationship between user stories (requirements) and development tasks is mostly straightforward: Just split up the user story into a few tasks.
A user story is not the same as a requirement. True, there can be some overlap depending on how 'high-level' the requirement is, but generally not the same. I get the impression that you're running into the same pitfall a lot of my former managers fell into: thinking about user stories simply as synonyms for "requirements", which is similar to when SVN users try to transition over to Git, but keep thinking in terms of SVN. (They then run into problems due to the bad starting assumptions.)
IMHO, a key difference between requirements and user stories is that requirements specify, in detail, how certain system components ought to behave; they're specifications that include inputs, outputs, assumptions/pre-conditions, possible exceptions raised, etc. They focus on what the system does.
OTOH, user stories focus on the expected outcome for the end-user without trying to create a detailed behavioral specification for system components. They focus on the expected user experience.
What I used to do, and this was a practice my team adopted, was to break down user stories into tasks. Your tasks could be as specific or vague as you wanted/needed them to be, but they were meant to be your progress indicators for actual work done towards getting the story to a done-done state.
Example
I roughly recall a US I worked on years ago where users needed to self-assign test cases so that everyone in the team was aware of which TCs they were working on to avoid duplicated efforts; the UI was a(n internal) web application. The user only saw a button, but the story was divided into several tasks that included some technical implementation details, etc.
User Visibility
But there is another type of application where most of the code has to deal with complex processing that is not directly visible to the user.
Is it possible to make it visible to the user in some way?
Consider a GPS. When you've missed your turn, you won't see the actual route re-calculation process, but the user does receive some useful feedback (e.g. "Recalculating...").
Compilers may display warnings or errors, or include new settings/options in the GUI for users to see that something new has been added. I'd think the users for compilers would be programmers, right? Wouldn't they see support for a new standard added?
While supporting a new standard would likely be at the feature level and would need to be broken down into user stories, have you made sure that, at least in some cases, you're not trying to use stories when you should be using features instead?
Image analysis in a car could be phrased in a way that lets the user know that the chances of ending up in a car crash have been reduced. For example:
As a passenger in a self-driving car, I need the probability of the vehicle causing an accident by crashing into an unrecognized object to be as close to zero as possible, so that I can travel more safely.
That US captures, at a high-level, things you'd normally have to specify using a combination of functional and non-functional requirements -including security, safety, etc.
However, a requirement might be more about the system; e.g.:
Function abc in component A must have the tolerance threshold value decreased in the image comparison algorithm to better detect objects moving slowly.
To me, this would easily be a task under the user story I mentioned above, titled something like: Decrease tolerance in function A.abc and then include other relevant details in it.
For a fluid simulation system, you could even have a progress bar that provides feedback about background tasks the system is performing, if this makes sense. (There's always a way to inform the user of something, though you may want to avoid being spammy.)
I don't know enough about the particular domains you've mentioned to come up with better and/or more realistic examples, but if there's a take-away here is that you can use different ways to provide user feedback about something less visible that the system might be doing, i.e. there might be ways to make invisible things a bit more visible. (Even if it boils down to writing a set of release notes that documents how much faster the system's performance is now due to your efforts, etc.)
Relationship between Stories and Tasks
Here it can get really difficult to relate tasks and user stories.
Our approach was to keep user stories focused on what the request was, why it was made, and what things needed to be true to consider the US "done". The how was always left out of the US and left to the developer(s).
The developer(s) would break down the problem described in the US into a set of tasks that they would work on.
I'm saying this as someone who, for the most part, did back-end server-side programming, which is probably as "invisible" as you can get for the end-user.
Depending on what I needed to do, I'd sometimes use AJAX to show a simple "loading..." animation/gif so that the user knew they needed to wait a bit for something else to complete, without getting the wrong impression. Sometimes it was as simple as this. A task for this would be appropriate.
Different Paradigm, Practicing, and Experience
Are there techniques to overcome this issue or is it just something we have to accept and make the best of it?
Beyond accepting the paradigm shift, practicing, and accrued experience, probably not much more to say. I often saw people trying to take shortcuts through the process. I'd advice against that, especially if you're getting started. As you get more experience, you can allow some flexibility, but avoid undermining yourself.
Given your prior wording, you're still thinking about stories as if they were "renamed requirements", which I think is a false assumption. I think this is a symptom of a deeper issue regarding the fundamental differences between Agile and non-Agile approaches.
Secondly, I think you should accept that agile is a paradigm shift compared to waterfall, which means that, while the process has similar goals, they go about it in very different ways. (Think SVN vs Git, if that helps.)
Try to improve your current understanding of the conceptual differences between requirements and user stories and accept they're not the same thing.
Learning from your Sprints - Retrospectives
What I cannot stress enough is the retrospective between Scrum Master and Developers at the end of each sprint. That is the place where they discuss things that "went well" or "didn't go well" in an honest/transparent way, and what doable changes will be implemented for the next sprint to address the "didn't go well" points.
That allowed us to adapt and even learn from each other's experiences, and before we knew it, we had improved significantly as measured by the general consistency of our team's velocity.
