Why is "Generate Language for Data Flow" step taking so long?

[Former Member]

I am building out a dataflow to read and xml (nested table xsd format) source via an XML_MAP transform and then some downstream query transforms.   It seems that when opening and/or saving the dataflow it gets very slow on the "generating language for data flow" step.

Is there any way to speed this step up?

[former_member187605]

Make your data flow less complex . Or improve your repository data base connectivity.

Any object defined in DS Designer is stored in the internal repository tables.  Many roundtrips to the database are needed for opening and saving a complex object. There's no way around.

Dealing with file formats based on large xsd's typically leads to this kind of performance decrease during development.

[Former Member]

Thanks Dirk!

Does the generate language step have to occur on both the execute and the save? 

If I run the job from the console then will the generate language step get bypassed?

What I have been doing is using the execute option in data services designer.

As for the designer, do you have experience on the improvement of the repository data base connectivity, what are your recommendations?

[former_member187605]

KInd of. When saving, your object definitions have to be writte to the database. When running, the definitions stored in the databse will be read out. It makes no difference if you run a job from the DS Designer or from the console. You'll always have a bit of overhead because of this operation.

Overhead is lowest when DS and database are on the same (big and powerful enough: cores, memory, disks) machine.