Feel free to comment here politically on a technical question

[Former Member]

As recently requested by at least two Coffee Corner habitues, I've posted an ABAP performance question down in the ABAP Performance Forum.

The question assumes (perhaps wrongly) that itabs in an ABAP program are swapped out when the app server scheduler swaps out the program itself. (This is the way iut works on the DBMS's that I'm familiar with.)

If this assumption is correct, then it is entirely appropriate for anyone to comment politically in this forum on the question of whether ABAP performance can be considered outside the context of anyone's general run-time environment.

[former_member181923]

James and Matthew -

When Dante wrote "The Inferno", IT hadn't been invented yet, so he thought hell only had 9 circles.

But now that we know it has a tenth circle, be careful you don't wind up in it.

Just because we have disk and CPU to burn doesn't mean we should burn it.

Even the American Baptists are coming to realize that we were created to use resources wisely, not waste them.

Regards

djh

[Former Member]

I think you guys have missed the point here.

Say it takes a week to fix a slow running program at a cost of typically 1000 EUR a day at current consultancy to end client billing rates in W.Europe.

Accenture (better known as "accidenture") or similar consultancy billings to the end client are currently around this level anyway so these costs are actually realistic.

At this rate we are speaking of around 6,000 EUR per person before tax for a week assuming Sunday is a rest day.

Now consultancies like "Accidenture" will probably have 2 or 3 guys working on this problem at a cost of around 18,000 EUR a week assuming that the rates quoted above which are about par for the course in W. Europe ( EU countries) are correct.

"Accidenture" employees of course won't get anything like this amount of money --I'm only considering here what the end client gets billed for.

We are now getting into the realms of SILLY money but it could still cost the end client up to 90,000 - 100,000 EUR a month if the problem takes even a little bit longer to fix than the original estimate.

You can get a whole slew of fast servers and HUGE disks these days for that amount.

I'm as previously posted not advocating sloppy code but is it really worth while spending a lot of man hours fixing stuff which might only be needed for a few months compared with just bolting in new faster hardware and solving the performance problem.

Unless you are in the banking industry still running IBM mainframe COBOL (Circa late 70's - 80's) most SAP applications change relatively quickly.

Cheers

-J

[ThomasZloch]

Hi David,

yes it can, because "ABAP performance" also deals with e.g. programming of efficient Open SQL statements or internal table access, which is platform-independent stuff.

The question you asked is technically more advanced than 99% of the stuff I see there, so at least that scares away the copy & paste camp. I hope you will receive some good answers (I don't know one unfortunately).

Cheers

Thomas

[former_member181923]

Thomas -

Yes - I agree that there are many important areas of ABAP performance that CAN be discussed independently of platform and run-time environment.

For example, a while back I posted an SDN blog thread on "FOR ALL ENTRIES" that actually got picked up by several non-SDN SAP bloggers in the general blogspace because it proved with actual stats that the performance of "FOR ALL ENTRIES" vs a select within a nest of loops is entirely dependent on what's in the where clause, even when you're using a perfect primary key or alternative index. (The reason for this is that when you overburden the where clause too many complicated "OR" strings, the optimizer laughs at you and goes right to a table-scan.)

This is an example of an ABAP peformance issue that CAN, as you say, be discussed independently of platform or run-time environment.

But the question of program-internal ABAP itab size as it relates to scheduler swapping CANNOT be discussed independenly of run-time environment, for the reasons I mentioned in my reply to Andrew in the original ABAP Performance thread:

Furthermore, consideration of this question (itab-size and swapping) leads immediately to a consideration of new-style ECC5+ Shared Memory Objects, as I point out in this follow-on thread here:

Because if it's true that ECC5+ Shared Memory Objects don't get swapped in/out by the scheduler like program-internal ABAP itabs, then any shop that doesn't like "selects within loops" should also be insistsing that their developers use Shared Memory Objects instead of program-internal ABAP itabs.

Turning now to the "political side" of the issue, you posted that the original question I raised down in the ABAP Performance forum is "more advanced" than 99% of the other posts in that forum.

But if you're correct about that, then your comment reflects a situation that SAP (and particularly SDN) should strive to rectify at every Tech Ed it provides for the next five years.

Otherwise, SAP is just as guilty as Bill and Larry of selling dangerous "vaporware" when SAP promulgates a construct like "no selects within loops".

Here's why.

Going back twenty years ago, EVERY developer was forced to consider the size of program memory relative to what the system could handle during scheduler-driven swaps.

But just like Bill and Larry hoped, most CIO's would now say that issues like program memory size relative to swapping are now completely irrelevant - because Bill and Larry's software is so slick that it "handles" all such problems for you.

So the end-result is that you have two kinds of SAP shops out there, as follows.

In the last three years, I've worked at two large shops (including my present shop) which try to enforce the "no select within loops" principle promulgated by SAP as a "best practice".

In the first of these shops, this principle was enforced as tightly as the restriction against "downloading" porn, and the "enforcers" didn't have the technical smarts to understand the "swapping" issues that result from using binary searches on large program-internal itabs to avoid selects within loops.

In the second of these shops (where I'm at now), the two relevant development "leads" are very technically savvy - if you raise the issue of swapping with them, they don't look at you like you have two heads.

So, if all shops were like the one in the second example above, then it would be OK for SAP to promuigate "no selects within loops" as a "best practice", because it would be generally understood that this is a general principle that may have to be violated (unless you go to Shared Memory Objects, or find a way to parallelize your program, or find a way to divide the work of the program into "chunks", etc.)

But since there are many shops out there like the one in the first example above, SAP is sometimes offering loaded guns to children when it promulagates a "best practice" like "no selects within loops" without mentioning some of the subtle consideratiaons that have to be taken into account when you try to apply this principle correctly.

Always a pleasure to chat with you ...

Best regards

djh

[ThomasZloch]

Hi David,

I hear ya, and I wish there was more "expert" content in the "expert forums". I can only talk about the ABAP forum though, not sure about the situation in other forums. There are always ongoing discussions about the "noise" and point hunting dilemma in a coffee corner near you.

Luckily there's still a bunch of real experts around that give good answers (or ask good questions for that matter). I have a growing "white list" of these folks for my own reference.

I met many ABAP developers over the years and many of them did not have much of a clue about memory usage or performance optimization in general, since you can nicely get away without these skills most of the time. The programs runs well according to spec in an almost-empty-just-gone-live box, and once the runtime gets longer and longer as the data volume grows, he/she is on the next project already writing the same poor code.

With this respect I think it would be a good exercise for every developer to actual spent some time in production support mode solving the problems that their own code caused.

That said, I will now look closer into that intersting "shared memory object" concept, thanks to your input.

Cheers

Thomas

[former_member181923]

Thomas -

I'm VERY glad that our exchange has prompted you to look into non-swappable SMO's. It takes the same time to build an SMO (more or less) as it does to build a large program-internal itab, but once it's built (as Thomas and rich point out so well in their new book), the program does NOT continuously import an SMO into program memory - it simply attaches to it and reads it.

To my mind, it is that feature of SMO's (coupled with their non-swappability) which makes SMOs the logical choice when you can't avoid large itabs (because you want to replace selects within loops with binary search reads.)

PS - Please don't think I'm "pimping" Rich and Thomas's new book when I suggest you get a copy ... in addition to a full treatment of SMOs that you can actually code from, there's a host of other neat stuff in there ... in my opinion, the SDN community owes them ...

Best regards

djh

[ThomasZloch]

In case you're referring to "Next Generation ABAP Development", it's very high on my wishlist and will be dealt with soon...

[Former Member]

I think here ---the wrong question.

Hardware is CHEAP and getting cheaper by the day for example I bought a replacement hard disk for a home desktop system last week -- 1TB and fast -- cost me 80 EUR. 20 years ago a 256 KB ( yes KB not even 1MB and certainly not GB / TB) 5 inch floppy disk system would have cost me maybe in today's money around 2,000 EUR.

People costs even in India, Bangladesh / Singapore etc. are getting more expensive daily.

I don't condone sloppy programming but sometimes is it worth optimising the last few bits out of code delaying a projects implementation at a huge cost when it's a LOT LOT cheaper just to get new hardware for a few cents more.

Most purists will die at this prospect -- but is it really worth spending time and resources on messing around with an application that actually works however badly the code is written when you actually compare costs of redeveloping the application compared with just bolting on extra hardware (when it's possible).

Cheers

-J

[matt]

>

> 
> Most purists  will die at this prospect -- but is it really worth spending time and resources on messing around with an application that actually works however badly the code is written when you actually compare costs of redeveloping the application compared with just bolting on extra hardware (when it's possible).

There's two issues here - one about getting a program to run as quickly as possible. The other about quality of programming. And both relate to manpower costs.

The former can lead to programs that are harder to read, harder to follow and therefore harder (and more expensive) to maintain and enhance. So it generally isn't worth trying to shave off a few more nano-seconds. And any time SAP introduce a technology that "removes the needs for programmers", you get applications that won't run as fast as if they were developed from scratch. E.g. FOX Formulas and Web Interfaces in BPS are inherently slower than ABAP Exit Functions, or an own-developed BSP. Or ABAP Queries are expected to be slower than a bespoke program. Unless the program really is time critical - other than writing it well in the first place, I don't think it's worth spending too much time.

The latter, however, is more debatable. Badly written code is difficult to maintain and every enhancement makes it more unstable. And therefore much more expensive to maintain - because in the long term it takes more time and resource. Most of the cost of development lies not in the original development, but in the support and maintenance. There's research that shows, in terms of Total Cost of Ownership, over the lifetime of a program, that a good programmer produces code that is some 20x cheaper than an average programmer. And 50x cheaper than a bad one. ( Which impacts the economic of off-shoring to cheaper countries. )

I recall one client I worked with, who had a extremely complicated commission calculating program. A new payment type was introduced and the program didn't work. The program was very badly written. We had to first re-write it properly, then run it in production and compare results with the original. Then we were able to find out what had gone wrong. And fix it. It took three weeks. It cost the company €500'000 as they had to pay commission at a level agreed with the unions, in order to avert a strike, which would have been devastating.

matt

[ThomasZloch]

I agree 100%. Golden cost/benefit balance should be found regarding performance. Once some basic principles are being observed during design/coding (e.g. using qualified keys to access DB or internal tables), it becomes more and more difficult to egde out more gains, e.g. using parallel processing and the likes. Increasing complexity, not worth it, most of the time.

I think a good example for the latter case is a wild mix of global and local memory (e.g. setting some global flag that is being checked 5,000 lines down the road...) and stretches of cryptic logic without any comments inbetween.

Sustainable code should be the goal.

On the other hand, repairing bad code and having the interface finish after 3 min as opposed to 3 hours (to the amazement of the IT manager) feeds me OK so far...

Cheers

Thomas

[matt]

Indeed. Bad code is a great source of my income as well.