IDoc vs APIs

With S/4HANA, everyone says API is the preferred approach for integration. I am trying to understand why older robust technologies like IDoc are no longer preferred, especially for an Inbound scenario.

Sorry, if this a dumb question. But, I would like to understand Why, if it is one.

Suppose a Sales Order creation in a system takes 3 sec via a BAPI (Time taken by IDoc processing, API processing, VA01 etc should be very similar to this, because they all go through the same processing logic in SAPMV45A)

When an external system calls S/4HANA to create a Sales Order via the API (https://api.sap.com/api/API_SALES_ORDER_SRV/overview), since this is a synchronous call, the call is handled in a dialog workprocess. This means that, throughout the execution of the logic in Sales Order creation, the dialog workprocess is held by the external caller, not releasing it to other users. According to the above assumption, that is 3 sec. In reality, most companies do not single thread their Sales Orders. This means, there may be multiple parallel calls into SAP, on various threads/dialog work processes to create Sales Orders. At some point of time, it could also result in all available dialog work processes occupied by middleware application, trying to create Sales Orders in SAP. At that time, business users logged onto SAP, will need to just wait for the slow system to respond.

Agreed that, SAP has provided parameters to control the number of workprocesses occupied by RFCs/a particular user etc. Suppose, the total available number of dialog work processes is 100 and the SAP administrator has setup a limit such that only 50% can be used by Integration user. This means, Integration engine can only use up to 50 dialog wps, which in turn means, it can only submit 50 concurrent Sales Orders into the SAP system. Mostly, threads on Integration Engine are built to run independent of each other. One thread on the Integration Engine need not know that there are already 50 connections active into SAP, and that it should Wait!! This could result in Integration Engine formatting all the data in correct format, go through every firewall to reach SAP, only to be rejected by SAP, because of the connection threshold.

Wouldn't it be better to use asynchronous Sales Order submissions into SAP? A great example is the robust IDoc framework. If Sales Order creation in a system takes sometime to go through the validation, lookups, rules, defaults, determinations etc., creation of an IDoc in the system happens in just milliseconds, because it doesn't go through any such steps. This means, the Integration Engine just submits the data and gets out of the SAP within milliseconds, freeing up the resources much quickly than using a synchronous API call. This also means, threads are available to exchange lots of data within a short period of time. Since IDoc data can be batched/packaged easily, multiple IDocs for many Sales Orders, can be submitted into SAP within a single call.

Talking in CRUD terms: I feel like, CRUD action performed by the Integration Engine in both scenarios are CREATE

API call: CREATE Sales Order (3sec synchronous call)

IDoc submission: CREATE a Request to Create Sales Order (few milliseconds asynchronous call)

When IDocs are received in SAP, SAP can leisurely process them according to its available capacity, and send confirmations back to the Integration engine, which can then pass it back onto the original caller.

There is another API based on SOAP (https://api.sap.com/api/SALESORDERBULKREQUEST_IN/overview) that SAP published for asynchronous A2A communication, but support only for limited sub-entities: header partner, header pricing element, payment plan item, item, item partner, item pricing element, and schedule line. oData API & IDoc support much more elements than this list. I haven't used this SOAP-API yet, hence not sure of how this handles the work process dilemma that I explained above.

Why is SAP still pushing everyone to use synchronous APIs, and why is there a broad idea floated around saying IDocs are obsolete (or atleast inferior to oData APIs) and should be avoided? Shouldn't this decision be absolutely based on whether the call should be synchronous/asynchronous? If there is no requirement for an synchronous response, why at all should the Integration Engine be allowed to hold the dialog work process?