IdM Database Copy Question

[santiobejero]

Hi Guys,

I just finished a DB copy of our Development System (Source) to Sandbox System (Target). I have a few set of questions and I hope you can clarify them:

1. The actual job which do the db table copy for MXMC_OPER user run for 8 hours. Considering that Production will be copied in the future, should I set the copy jobs as "Bootstrap"? This is that I expect running this to production should exceed to 12 hours and dont want to repeat the process as a downtime of the source (Production) is required on this activity.

2. I have seen the Runtime Engines are copied as well from the source, is it okay to delete them? Or it will cause any implications in the system newly refreshed?

3.Should I run a DB and Windows Instance restart once the actual copy was done? I am just thinking that it might be a good practice to do just to refresh any gears or flow that might got affected by the copy?

4. Any other best practices you can recommend that was not covered in the documentation but you encountered and apply during your actual db copy?

Thanks in advance as always.

Best Regards,

Santi

[Steffi_Warnecke]

Hello Santi,

since I had the great fun of doing a database migration (for 7.2) last year (for which you use the copy job, too), I have some input:

I can answer No.1 with a clear YES.

More then that, check the Java options of the syscopy dispatcher.

I had a lot (and I mean A LOT) of issues with my production copy job restarting time after time (after time...), because the heap space wasn't big enough. I ended up setting it to 8 GB at the end and then I still had to manually help the job along by disabling passes in the copy job that were already sucessfully done.

Are we talking about a 7.2 or 8.0 installation here?

.

Regards,

Steffi.

[santiobejero]

Hi Steffi,

Thanks a lot for the confirmation on point #1, I was as well under the impression that it should be set as bootstrap; considering the processing time it incurred in just a copy of our Development to Sandbox. By the way, we are just simulating the steps that we will do in production. So the actual fun (and I hope the type of fun is resolvable and not gonna bug me a lot) should happen in a couple of weeks once we do this in Production going to our Sandbox system.

For the Java Options, what does it do relevant to the Database Copy? I might be missing some basic knowledge perhaps? 😞

In addition to this, when you say "I still had to manually help the job along by disabling passes in the copy job that were already successfully done.", does that mean the job failed couple of times and you need to re-run them?

I have another question by the way, I have check the job that copies the OPER related data (Job Name: 06. Copy OPER tables - [MSSQL_MSSQL_SYSCOPY]), it shows errors on specific tables such as "Pass: Copy table DBO.MXP_EXT_AUDIT" but it doesnt show much of what is the error it just says error, do we have any way to analyze what might happened? Or we can just ignore those error as those might just be errors saying some entries can't be copied?

I appreciate you've mentioned about heap size of the Production DB, i will check this with our DB team if the Target system has enough space to copy the DB of Source system.

Appreciate your time and effort in confirming these items.

Best Regards,

Santi

[Steffi_Warnecke]

Hello Santi,

soooo...

Java options: It's like memory cache. At least that's how it translated for me. You can set it for each dispatcher and that is what they use/reserve from the server to do their job. Again: that's my crude understanding. Not setting it for the syscopy dispatcher (and then not setting it high enough) resulted in it re-starting again and again (sometimes after 14 hours and sucessful processing of a lot of entries).

The job didn't fail. It just re-started and began again from the top or stopped doing anything (with status "running"). So I started to check the tables in the source and target database by hand and if the content was the same (entry count via sql query), I disabled the passes for that table in the copy job. Then I started the job again.

Yes, we had that issue, too. It's entries with "$function" in their values. Those were not copied (same with constants btw, so be careful if you use scripts in there). We had this for these tables:

• MC_GLOBAL_VARIABLES
• MXI_OLD_VALUES
• MXI_VALUES
• MXP_AUDIT
• MXP_EXT_AUDIT

"MC_GLOBAL_VARIABLES" contains the global constants, the name is misleading.

The values were a result of old, faulty data, were the values should be created via scripts and something went wrong (so the scriptname shows up). We didn't copy those over, after we knew what was happening. I saw it as a kind of data cleaning. ^^

You can check with this code

select * from MXI_OLD_VALUES 

where avalue like '%$FUNCTION%'

The results were exactly the error count in the job log for each table.

.

With heap space I meant the available RAM for the java options. But it can't hurt to be sure, there is enough space for the data. 😉

.

Regards,

Steffi.

PS: And you're welcome. Feels like therapy talking about these things. 😄

[santiobejero]

Hi Steffi,

Thanks for the response, apologies for the sooo late response with you. I was caught with deliverables and wasnt able to login again on the community.

Anyways, I have checked and the queries provided and I can confirm that you are 100% correct. I tried this and get the same total of number, so I believe I can at least disregard this with my current prototyping phase of this db copy, but somewhere down the line, of course needs to take account at least as well. 🙂

For the java option, should i put "-Xmx2g"? Does this value a universal value at least or we have certain specific values for different specific scenarios?

Thanks as always & Best Regards,

Santi

[Steffi_Warnecke]

Hello Santi,

"-Xmx2g" means "2 GB" (the "2g" at the end). That's what I have in my currect setup for the normal dispatchers (provision, batch etc).

Like I wrote in my first reply, in the syscopy dispatcher I ended up using 8 GB. If I remember correctly you can use something like 80% of the available RAM. The RAM of the source system that is, because that's where the dispatcher is running. So just check your system values and adjust the java options accordingly.

.

Regards,

Steffi.