UDF for Record Splitiing.

Hi Experts,

I am new to java UDFS can anybody give the UDF for record splitting, we have requirment that ..our inout is bulk set of records for EX ; 40000 we need to split them into 5000 record packets and process them(calculate no of records and send).

Thanks in advance.

Srinivas

Edited by: sri_rambo on Aug 20, 2009 6:27 PM

Edited by: sri_rambo on Aug 20, 2009 6:30 PM

SAP Managed Tags:
SAP Process Integration

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Accepted Solutions (0)

Answers (2)

Hi Sri,

No need to use the UDF for records split.

If you are using a file, then you can handle it in CC -> records per message.

Even a Script can be used to split the file and PI can read.

By the way, whats the message size and whats the value you have given in EO_MSG_SIZE_LIMIT?

Regards,

Neetesh

Edited by: Neetesh Raj on Aug 20, 2009 11:04 AM

Edited by: Neetesh Raj on Aug 20, 2009 11:05 AM

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Hi Neetesh,,

Its a JDBC channel we using and JDBC channel scheduled to trigger the DB and pull the data records only once in a day so we decided to split the records and process.

Any further light.

Thanks,

sri

Hi Sri,

UDF won't solve your problem as the code in UDF works on the records which are in the queue in PI. And you want to chunk the records at the source system itself.

So, you might have to write a module at adapter-level. This can be implemented through an EJB, by deploying it on to the J2EE server.

Else you can check with your Database team, if they can chunk the data at their end.

Regards,

Neetesh

Sri,

One more thing.

Add one more EO_INBOUND_TO_OUTBOUND, this will allow data not to be placed on outbound queue and will push it to the inbound queue.

The advantage of this is when you are processing high volume data, you get a better performance. But the disadvantage is that, if there are no receivers (for e.g.), the inbound queues gets blocked.

Regards,

Neetesh

Hi Experts,

Any other suggestions.

Thanks in advance.

Hi,

Could you explain your requirement in more detail?

when you say 40000, what are you talking about?

With a examples we can understand in a better way what you want

Regards

Ivan

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Hi Jose,

We have an a interface in which XI pulls data from DB and post that data into another DB here the incoming records to XI are very bulk around 30000-50000 so we decided to split the records and process for this in mapping to split this messages we need a UDF.

Thanks for your response please advise the UDF.

Sri

Hi sri_rambo,

Once you have the message in XI You are not able to generate other messages(XI messages). I understand You want to split the message and generate other messages to put in the DB destination. One solution is develop your custom adapter(It is to hard) and the other option is in DB origin splitting the messages.

Regards

Ivan

Hi Sri,

Probably you can come up with a query which picks only 5000 or 10000 records at a time and send to xi instead of reading the whole data.

Regards,

---Satish

Hi Sateesh,

To select 5000 records while selection statement we need to maintain one more coloumn in table but my user not ready to create one more coloumn in DB table so we decided to map and split records in mapping.

Any othere suggestions experts.

Appretiate if any java programmer is ready to wirte an UDF on these. Thanks In Advance.

Sri

Hi sri,

For this no need of UDF ...You can achieve this by normal Graphical mapping....

input------>index[intialvalue 1, increment 1,count indices  context]---->div-------->ceil----->splitByValue{valueChange}---->collapsecontext---->target
                                                  Constant[5000]--------/

Thanks.

Hi,

Chek this blog [Chunking of IDOC segments - Using XSLT|https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/10379] [original link is broken] [original link is broken] [original link is broken];. XSLT mapping can help you. Sravya has a comment in the blog to do the record split using node function and a UDF...

~SaNv...

Hi Jyothi,

Thanks for your prompt response, but can yopu bit clear i am not geeting where you are defining constant((5000) in above logic.

THanks in advance.

srinivas

Hi,

Constant is a standard function in Graphical mapping..It is there in Constants...Double click on that function and give the value 5000....

Forther Quries getback...

Thanks.

Hi Santhosh,

Thanks for your prompt response but my team is not good in XSLT mapping, yes sravya indicatred that node function and UDF combop will solve problem ..we also decided same way but aunable to write a logic in UDF ..we belive splliting can happen in UDF .

Thanks

Sri

Hi Jyothi,

yeah constant is there in graphical mapping we are aware but in the logic given by you where you defining the condition if count is equal to 5000(constant) then process the records.

Thanks,

Sri

Hi sri,

Assume like for input field suppose 40000 records are coming....and i am giving indext to those records, by incrementing index by 1...and i am dividing that index by constant value 5000 and ceiling that....and Spliitting by value change and Collapse context...So this will split it...

I have done the similar kind of scenario...

Like I am getting 10 Records form my Source side...

input is like this...

Record1
Record2
Record3
Record4
Record5
Record6
Record7
Record8
Record9
Record10

But in my target Structure 3 records has to be there per 1 Message type...

like..

MT_Target..
Record1
Record2
Record3

MT_Target..
Record4
Record5
Record6


MT_target
Record7
Record8
Record9

MT_Target..
Record10

I think your also like similar type...

Here I am giving index to each Record...So I will get inde like 1, 2, 3.....10 and i will divide it with 3..so 0.333,0.666,1,1.33....after ceeling and splitBy value and context collapse context it will come only 1,2,3,4 only....So if i map this to MT_Targer..I will get 4 MT_Target...

I hope you are clear now...

This will Surely work if your Mapping is similar kind....

Use that...

Hi Jyothi,

Thanks For clarity ..yeah my requirment is just like urs only ..yes by using your logic we did half a work now but one more doubt is my source MT is 0..unbounded and for target we changed occurance for 0..unboubded for requirment demanding then is this logic will be worked out.Let me knkow if you need any further details.

Thanks,

Srinivas

Hi Sri,

It will surely work..

The logic I have given on my previou reply,map that to Your target Message type ...If you want more, provide me the complete input and output message type and tell then I will provide you the total mapping...

Thanks.

Thank you jyothi,

Let me try then can you help me out that ..how you used error handling while doing this issue if any of packet got failed to insert into JDBC then how that exception was handled..Because if any of packet set was not processed we need to know about that packet.

B'rgrds,

Srinivas

Edited by: sri_rambo on Aug 21, 2009 9:36 AM

Hi Jyothi,

me like for input field suppose 40000 records are coming....and i am giving indext to those records, by incrementing index by 1...and i am dividing that index by constant value 5000 and ceiling that....and Spliitting by value change and Collapse context...So this will split it...

I have done the similar kind of scenario...

Like I am getting 10 Records form my Source side...

input is like this...

Record1

Record2

Record3

Record4

Record5

Record6

Record7

Record8

Record9

Record10

But in my target Structure 3 records has to be there per 1 Message type...

like..

MT_Target..

Record1

Record2

Record3

MT_Target..

Record4

Record5

Record6

MT_target

Record7

Record8

Record9

MT_Target..

Record10

i am not getting this type of output when i m using the logic provided by you getting

MT_Target

as an output my source structure is like this

Message1

MT_messagetype

Statement_

Row

field1

field2

field3

fieldn

and target structure is

Message1(node)

MT_messagetype(node)

Statement_(node)

tablename(node)

action

table

Access(node)

field1

field2

field3

fieldn

here i am counting the ROW field and applying the logic and mapping to the target MT_messagetype(node) or should i map to Access(node)

Thanks,

Srinivas

Hi Sri,

I think you have used SpliByValue(Each Value) Thats why it is creating 10 times...

Use SpliByValue(ValueChange)....This is very important step in this mapping...

Thanks.

Hi Jyothi,

I am using splitbyvalue(value changed) only then collapseContext then target.

please check my message structures that to which field i need to map... to MT or Access at target side.

Thanks in Advance.

Srinivas

Hi Sri,

have you mapped like this...

Row--->index(1,1,Context)----->div------>ceil----->SplitByValue(Value Change)----->collapseContext-------->MT_Target

Now also I have tested with the same test case keeping 10 Records...It worked, It has created 4 MT_Target ' s for me....

check your Mapping one more time....

Thanks.

Provide your Structures properly.....and tell me what is the input to your mapping....

Provide some sample structure to execute....

I have tested with my scenari like...

MT_Source   0..unbounded
   Item           0..unbounded


MT_Target   0..unbounded
  Item             0..unbounded

I have mapped like this...

Item--->index(1,1,Context)--->div----->ceil----->spliByValue(Value change)---->collapsecontext----->MT_Target
                  Constant(3)---/

Item--->index(1,1,Context)--->div----->ceil----->spliByValue(Value change)---->Item
                  Constant(3)---/

I am giving Item values as Record1, Record2....Record10 for source....

and I am getting Ouput like ..

MT_Target
  item  Record1
  item  Record2
  item  Record3
MT_Target
  item  Record4
  item  Record5
  item  Record6
MT_Target
  item  Record7
  item  Record8
  item  Record9
MT_Target
  item  Record10

Check with these...

Thanks.

Hi Jyothi,

I am using the same logic provided by you in index when i am using CONTEXT its not counting the values in display queues its showing

like this its now counting but if i give entire quese in index then its counting but reocrds are not coming in a way that we expected and you are saying collapscontext after splitbyvalue so only one MT is coming at output under that the num of Access nodes are generating, but where our req is Access nodes should be spilt into respected MT tabs right for example if we give 10records and split at 3 then 4 MT should be generate with 3,3,3,1 W.R right,

please revert back me

Thanks

Srinivas

My Source and target structure are,

Message1

MT_messagetype0..unbounded

Statement_

Row0..unbounded

field1

field2

field3

fieldn

and target structure is

Message1(node)

MT_messagetype(node)0..unbounded

Statement_(node)

tablename(node)

action

table

Access(node)0..unbounded

field1

field2

field3

fieldn

Thanks,

Srinivas

Edited by: sri_rambo on Aug 21, 2009 2:35 PM

Thanks Jyothi,

I got it we now solved the problem but we need some help in Error handling for this process suppose if one packet failed who that could be maintained in runtime.

Appritiating your help. THanks in advance.

Srinivas

any ideas for error handling of this concept experts we are using this in BPM.

Thanks In Advance.

B'rgrds,

Srinivas

Edited by: sri_rambo on Aug 21, 2009 5:04 PM

Ask a Question

Top Q&A Solution Author

User

Count

101