How to know “all these things”

There is one predominant reason for why databases and database technology get a spot in the conversation of “the business”: performance.
Understanding system performance and coming up with ideas for improving it typically requires a form of end-to-end understanding with different levels of detail insight on how a query from a client application is processed by SAP HANA. Before long, one looks into “explain plans“, “execution paths”, SQL, PlanViz diagrams, data distributions, table definitions, stack traces and even NUMA-process management on Linux/Hypervisor-level. That’s a lot of concepts and layered technology and most of those concepts come with a development history of at least 30 years if not more.
This means that prior knowledge is not only implied but required to get a proper understanding of what concepts are put to work in your specific setup. Intuition about how the system works is not your friend here.

But where do you learn about all these things? The product documentation? A university SQL 101 class? Blog posts from strangers on the Internet? Books? Or a mooc with a certificate?

Personally, I’m always happy to find good collections of material, maybe even curated into topics. One of those collections is The Red Book (Readings in Database Systems), which covers all important topics of Database Management Systems. It even features a chapter “3. Techniques Everyone Should Know” in case you really only want to have a quick overview.

I read about this awesome (and free) online book in a blog that I can only recommend: The morning paper by Adrian Coyler. He linked to the red book in his post (Database) Techiques Everyone Should Know.
There are also many more recent posts about database related papers like Learned Index Structures or the design and implementation of modern column-oriented database systems which explain current developments in DBMS technology that will probably end up as a feature in one of the big DBMS systems. The great thing about Adrian’s blog is that it is a kind of Cliff’s Notes for computer science papers. One does not have to actually read the paper to get the core ideas; but Adrian always makes the paper available to read for oneself.
While I mention this blog, I like to say “Thank you!” to Mr. Coyler whom I never met, but I certainly benefitted a lot from reading his outstanding blog.

A long comment

Last Monday I read DJ Adam’s blog post about GraphQL and how this tool might be useful to be added to the toolbox of SAP UI5 developers.
There is a lively discussion over the blog’s topics going on and most of it was around the GraphQL technology and its potential usage in UI5.
I added a comment to this and Mr Adams used this to hook me in for a blog post.
So here we are.

As someone that read through the discussion somewhat uninvolved since I am not doing any front-end development what stroke me was the narrow focus of it.
Why was there no question about how this added tool would likely make a change to the development projects and teams? Reading a bit deeper in the linked blog post from Jeff Handley a lot of the arguments are not about technology, features and implementation details but about how the tool is helping the product development teams to do their jobs better.
To me, it seems like a rather typical SAP technology discussion, where “THE TOOL”, “THE PLATFORM” or “THE SUITE” is the bit that will make the difference between successful developments and the other ones.

Now, while I am a sucker for great tools like the next person, I am in the corner of folks that believe “it’s the photographer that makes the great picture – not the camera“. Technology can help with many problems but if one does not have a clear understanding what problem needs solving simply picking up the lastest-greatest-shiniest tool likely means the wrong tool is used and/or used in a wrong way.

One example I mentioned in my comment hinged on the “declarative” nature of GraphQL. I do not know how GraphQL works specifically but I have some experience with how declarative programming works in many SAP shops.
The main declarative language present in these organisations is SQL. ABAP programs use it, SAP HANA models have it (in some way) and the analytics teams are constantly building “queries”.

My point is that this typically does not work too well.
Invariably I come across data models and queries that show the developers tried to “tell the database what to do“. And that even though the very same developers commonly do neither have a proper understanding of how the database works nor how to understand what it actually does in the first place.
Still, “that index needs to be used!” and “this hint is crucial for the performance“!

I cannot claim to know for sure why this is, but I got some suspicions:

  • often ABAP/SAP developers moved into their roles horizontally from either the “tech support” or from the “business support” team. That means great familiarity with the organisation’s landscape and processes but does not guarantee a firm grasp of the concepts implemented in the technology.
  • education and learning of abstract concepts is not a common activity in SAP development teams as there is no immediate payoff.
  • the official SAP technology documentation and training is focussed on functions and features and actively avoids concepts and broader ideas.
  • training content often focusses on simplistic examples leaving out real-world complex scenarios

This is not to say that it is the development team’s fault – but a lot of the IT technology that SAP has embraced in recent years requires a comprehension of the underlying concepts. Most of those concepts tend to be more involved than computation “mechanics” and cannot be intuitively understood correctly.
“Learning by example”, which I would bet is the most prevalent learning style in SAP environments, only goes so far if one is trying to get a hold on new non-trivial ideas.

Coming back to the GraphQL discussion to me this looks like another tool that will be “used the wrong way without error messages” if at all. Will the majority of development teams be able to realize the same benefits as the Concur team? I doubt that.
Will the choice of tool make a difference to the outcome for the organisation in a way that makes a difference to the organisation? Probably not.

If one squints now, it is pretty obvious that this is the “full-stack developer is dead” argument from some years ago.

So why would I be putting this into the ring here? Because it holds true so well – especially with SAP tech environments.
We (SAP) have tried to hammer in the idea of “push down” and “processing where the data sits” for years now, still most teams I encounter think “business logic” == “ABAP/JavaScript” and subsequentially get surprised when the DB access does not deliver the expected speed ); after all it’s the new tool, HANA, right?
Understand that this is just one example of the issue here.
It could have also been ABAP OO, SOAP architecture, or Virtual Data Models. Common with all these tools is that they are going to be used by the very same people that used to do “things the proven way” for many good years.
Common also is that the creators of those tools usually have a very different view and probably assume too much conceptual overlap with their audience. If you ever listened to one of the gurus of “Graph DBs”, “Docker” or “Haskell” you probably know that feeling of conceptual dissonance from your day to day work.

This gives a lead into another aspect of the discussion around DJ Adams’ blog post: the argument that it would “help to get more developers to SAP platforms” if GraphQL (or any other currently popular tool) would be available.
When I read claims like that I wonder if there are any organisations that use IT as part of their established infrastructure and that then go and swap out major parts of it, because some developers like the new parts better.
In my personal experience that never happens. Sure, the new marketing hero gets the reporting frontend s/he needs to be successful – but that is closely tied to tangible results.
If developers cannot show how the end result will be (very) positively affected in concrete, preferably short terms, then existing tools tend to stay in place.

Change is hard, slow and expensive and the ticket for this journey really must be worth it. And commonly the new lands look a lot nicer from a good distance.