cancel
Showing results for 
Search instead for 
Did you mean: 

Intermittent APPCRASH when opening a form with CrystalReportViewer

Former Member
0 Kudos

We have an application that, as part of the functionality, allows users to generate and preview a report. It uses CR for VS 2010. The report preview dialog is a Windows Form that includes a CrystalReportViewer.

This functionality works, most of the time, but we are observing intermittent APPCRASH/Stackhash crashes when this form is opened of this nature:


------------------
Problem signature:
  Problem Event Name:  APPCRASH
  Application Name:      ****
  Application Version:    ****
  Application Timestamp:           4dd40fa4
  Fault Module Name:    StackHash_7698
  Fault Module Version:  6.1.7600.16385
  Fault Module Timestamp:         4a5be02b
  Exception Code:         c0000374
  Exception Offset:        00000000000c6cd2
  OS Version:    6.1.7600.2.0.0.256.48
  Locale ID:       1033
  Additional Information 1:          7698
  Additional Information 2:          7698c42b9ee8da1ebad3b3c9521cacfd
  Additional Information 3:          ad41
  Additional Information 4:          ad418ba72161d4cd11d7688fd368f113

This crash occurs intermittently. We have an automated script that tests the GUI and it may run for 5 minutes or several hours before encountering this error, with a handful, to tens, perhaps over a hundred report generated successfully.

As we understand it, the error code c0000374 means:

// MessageId: STATUS_HEAP_CORRUPTION

//

// MessageText:

//

// A heap has been corrupted.

CR is not the only part of our application that uses unmanaged memory, but this crash consistently happens when opening the dialog with a CR component in it (which renders a report).

I realize I've left out a lot of potential information, but will add more upon request. This problem is causing us quite some headaches and I just wanted to see if anyone else has experienced something similar.

Any ideas?

Accepted Solutions (1)

Accepted Solutions (1)

former_member183750
Active Contributor
0 Kudos

1) I'd be interested in a bit more detail on the 'crash". What actually happens? Does the app hang? Does the app unload? IS there an error thrown on the screen? E.g.; what would I see experience if I was sitting in front of that computer? Make sure you are using SP 1 for CRVS2010:

[original link is broken]

MSM

http://downloads.businessobjects.com/akdlm/cr4vs2010/CRforVS_mergemodules_13_0_1.zip

MSI 32 bit

http://downloads.businessobjects.com/akdlm/cr4vs2010/CRforVS_redist_install_32bit_13_0_1.zip

MSI 64 bit

http://downloads.businessobjects.com/akdlm/cr4vs2010/CRforVS_redist_install_64bit_13_0_1.zip

2) Are you able to reproduce the issue on your development computer (without using a script - see (7) bellow)?

3) Is this a 32 bit or a 64 bit app?

4) Make sure the app is not compiled as "Any CPU"

5) What is the OS?

6) What is the database (or are you using datasets) and how are you connecting to it (ODBC, OLE DB, etc.)?

7) Re.: "We have an automated script that tests the GUI and it may run for 5 minutes or several hours before encountering this error, with a handful, to tens, perhaps over a hundred report generated successfully."

Can you generate the issue without the automated script? Is the "crash" behavior different if you do not use the script?

😎 Post code used. There is a limit of about a 1000 characters in these posts before you loose formatting, so if need be, split the code over two or more (hope not) posts...

Ludek

Follow us on Twitter http://twitter.com/SAPCRNetSup

Got Enhancement ideas? Try the [SAP Idea Place|https://ideas.sap.com/community/products_and_solutions/crystalreports]

Former Member
0 Kudos

Hi Ludek, thanks for your response.

1) I'd be interested in a bit more detail on the 'crash". What actually happens? Does the app hang? Does the app unload? IS there an error thrown on the screen? E.g.; what would I see \ experience if I was sitting in front of that computer?

Basically, an appcrash dialog, from windows is presented to the user. I don't have a screenshot from us, but this is the dialog I'm referring to:

[http://img63.imageshack.us/img63/4180/27355960.jpg]

The user is forced to close and restart the application. In our application, we catch all unhandled exceptions, so I can state that no exceptions are thrown (which would agree with the idea that this is a problem in unmanaged code).

Make sure you are using SP 1 for CRVS2010:

If we can pinpoint that this update will solve the problem, or have exhausted other avenues, I will of course try this. However, our application runs in highly regulated environment where such a dependency change is significantly more costly than other solutions (changes to our own application, etc.). So if you'll bear with me, I'd like to investigate some of the other aspects first, such as the ones you've suggested below.

2) Are you able to reproduce the issue on your development computer (without using a script - see (7) bellow)?

Not yet. Unfortunately our development systems are not a close reproduction of our production systems. For example, dev systems receive Windows updates, while production systems are installed with an image and do not receive updates.

3) Is this a 32 bit or a 64 bit app?

x64. All of our systems are built here and are x64, so we control the platform.

4) Make sure the app is not compiled as "Any CPU"

Currently we are indeed compiling as "Any CPU". Could you elaborate for me why we want to avoid doing this?

5) What is the OS?

Windows 7 x64

6) What is the database (or are you using datasets) and how are you connecting to it (ODBC, OLE DB, etc.)?

SQL Server CE v3.5 SP2, OLE DB

7) Re.: "We have an automated script that tests the GUI and it may run for 5 minutes or several hours before encountering this error, with a handful, to tens, perhaps over a hundred report generated successfully."

Can you generate the issue without the automated script? Is the "crash" behavior different if you do not use the script?

We have not manually produced it here, to my knowledge (we have very limited manual testing resources), but it is occurring in the field, as we have received reports from some of our users.

To my knowledge, the behaviour is the same, although we are relying on verbal communication of the error in most cases.

😎 Post code used. There is a limit of about a 1000 characters in these posts before you loose formatting, so if need be, split the code over two or more (hope not) posts...

The code is a bit too diffuse and I am not yet aware of what the culprit is for me to post a relevant snippet. The crash occurs when we call myReportPreviewDialog.Open(), where myReportPreviewDialog is a Winform that includes a CrystalReportViewer which has been configured to render a report.

I'll await your response on the x64 issue, because I think this would be the easiest one to rule out.

Thanks!

Zamir

Former Member
0 Kudos

Make sure you are using SP 1 for CRVS2010:

If we can pinpoint that this update will solve the problem, or have exhausted other avenues, I will of course try this. However, our application runs in highly regulated environment where such a dependency change is significantly more costly than other solutions (changes to our own application, etc.). So if you'll bear with me, I'd like to investigate some of the other aspects first, such as the ones you've suggested below.

On closer inspection, I do believe we are using this version. The versions of the CR DLLs that we are building against are all 13.0.2000.0 and the runtime we have installed is the same as the one you linked for me. Sorry for the confusion.

former_member183750
Active Contributor
0 Kudos

Hello Zamir.

Many thanks for the great answers (I wish I could post this thread as an example to all ).

Anyhow.

Let's start with SP1. You may already be using it. Check the version of the crpe32.dll (C:\Program Files (x86)\SAP BusinessObjects\Crystal Reports for .NET Framework 4.0\Common\SAP BusinessObjects Enterprise XI 4.0\win64_x64). It should be 13.0.1.220, if it is, we're on SP1. If it is not, understanding your concerns, going to SP1 will still be the DE thing to do. E.g.; I will not do any testing unless you are on SP1 (sounds bad I know, but just laying the cards on the table - face up )...

The fact that you are not able to reproduce the issue on your development system my actually be quite encouraging and perhaps a starting place for us (given the SP1 conundrum). It will be interesting to compare the dlls (both Crystal and database client). Normally the [Modules|https://smpdl.sap-ag.de/~sapidp/012002523100006252802008E/modules.zip] utility is very useful for comparing dlls loading on a computer that works and one that does not. As this is a 64 bit app, it will be a bit trickier. Modules works really well on 32 bit apps. On 64 bit apps it does not save the log. You can still see it, but not save it. Anyhow, I am attaching a 64 bit version of Modules. You will have to check the versions of the CR DLLs and database client DLLs by looking at the logs and writing the info down. Again, I realize this is a difficult task on a customer system, but...

Re. apps compiled as "Any CPU". These should work - as long as only the correct "bitness" CR runtime is deployed. E.g.; On a 64 bit OS, do not deploy both a 32 bit and a 64 bit runtime. This may cause issues. My recommendation in the previous post was a rather blanket recommendation that quickly eliminates all kinds of confusions for many people.

Re. the database - SQL Server CE v3.5 SP2, OLE DB. Make absolutely sure the db client is the same on the problem computer as it is on your development computer. Use Modules for that. If htere is a later SP, apply that. I can not overstate the importance of this.

Re.; "To my knowledge, the behavior is the same, although we are relying on verbal communication of the error in most cases."

It will be important for these users to let us have screen shots of the issue as they get it. I realize this may be an inconvenience to the customer, but what is essentially a hearsay evidence from what is typically a non technical user is always something to take with a grain of salt.

Re. code. One thing to be absolutely sure of is that you are using .Close and .Dispose on the report objects as the user is done with them.

One other possible source of issues; printer drivers. Specifically printer drivers not designed for the particular OS. Ensure that the printer driver(s) used by the customer on that WIN 7 are designed for WIN 7 - even if they are only viewing and or exporting. CR has a large dependency on printer drivers.

- Ludek

Former Member
0 Kudos

Hello Zamir.

Many thanks for the great answers (I wish I could post this thread as an example to all ).

Same to you. Debugging issues like this is never fun, but I know from experience that clarity from both sides helps us get to a resolution

Let's start with SP1. You may already be using it. Check the version of the crpe32.dll (C:\Program Files (x86)\SAP BusinessObjects\Crystal Reports for .NET Framework 4.0\Common\SAP BusinessObjects Enterprise XI 4.0\win64_x64). It should be 13.0.1.220, if it is, we're on SP1. If it is not, understanding your concerns, going to SP1 will still be the DE thing to do. E.g.; I will not do any testing unless you are on SP1 (sounds bad I know, but just laying the cards on the table - face up )...

I checked both my development machine and a production system that I have access to in-house (which is able to reproduce the problem via a script). Luckily for me, both are at version 13.0.1.220.

The fact that you are not able to reproduce the issue on your development system my actually be quite encouraging and perhaps a starting place for us (given the SP1 conundrum). It will be interesting to compare the dlls (both Crystal and database client). Normally the Modules utility is very useful for comparing dlls loading on a computer that works and one that does not. As this is a 64 bit app, it will be a bit trickier. Modules works really well on 32 bit apps. On 64 bit apps it does not save the log. You can still see it, but not save it. Anyhow, I am attaching a 64 bit version of Modules. You will have to check the versions of the CR DLLs and database client DLLs by looking at the logs and writing the info down

I'm having a bit of trouble with the Modules utility. I launched it and when I get it to generate a list, I cannot see the process for our application in the list. It is running and appears in the task manager, but it is not listed under the list of processes, nor do I see any of it's loaded DLLs. Any ideas?

Re. apps compiled as "Any CPU". These should work - as long as only the correct "bitness" CR runtime is deployed. E.g.; On a 64 bit OS, do not deploy both a 32 bit and a 64 bit runtime. This may cause issues. My recommendation in the previous post was a rather blanket recommendation that quickly eliminates all kinds of confusions for many people.

Thanks for the clarification. I am going to stick with 'Any CPU' because I know that we only deploy the x64 runtime and if I switch to explicit x64 compilation, Visual Studio reports some false-positive warnings that I'd rather not have (it's a bug in Visual Studio).

Re. the database - SQL Server CE v3.5 SP2, OLE DB. Make absolutely sure the db client is the same on the problem computer as it is on your development computer. Use Modules for that. If htere is a later SP, apply that. I can not overstate the importance of this.

As above, I could not use modules, but I did confirm the DLL versions in their respective Program Files location on both machines and they were the same (3.5.8080.0).

It will be important for these users to let us have screen shots of the issue as they get it. I realize this may be an inconvenience to the customer, but what is essentially a hearsay evidence from what is typically a non technical user is always something to take with a grain of salt.

I just talked to one of our field engineers and he has seen screenshots to confirm it is the same issue as the one we can reproduce in-house.

One thing to be absolutely sure of is that you are using .Close and .Dispose on the report objects as the user is done with them.

We do, in general, make these calls and recently have gone through our code for memory leak possibilities, so this was recently checked for by developers.

One other possible source of issues; printer drivers. Specifically printer drivers not designed for the particular OS. Ensure that the printer driver(s) used by the customer on that WIN 7 are designed for WIN 7 - even if they are only viewing and or exporting. CR has a large dependency on printer drivers.

We use the Samsung CLP-320 and the driver package it installs are "Win 2000/XP/2003/Vista/2008/Win 7(32,64bit)", so I assume the correct version gets installed on our Win 7 x64 system.

Former Member
0 Kudos

I don't know if it's relevant, but I should add that for our SQL Server CE, we install both the x86 and x64 runtimes, as per instructions from Microsoft:

[http://www.microsoft.com/download/en/confirmation.aspx?id=5783]

On a 64-bit Computer, install both the 32-bit and the 64-bit version of the SQL Server Compact 3.5 SP2 MSI files. Existing SQL Server Compact 3.5 applications may fail if only the 32-bit version of the MSI file is installed on the 64-bit computer. Developers should chain both the 32-bit and the 64-bit MSI files with their applications and install both of them on the 64-bit Computer.

former_member183750
Active Contributor
0 Kudos

Re. Modules. My bad. I promised I'd attach the 64 bit version here and I forgot.. so here it is now.

Re. the 32 and 64 bit client, I think that should be ok, but I'll confirm on Monday.

Re. the printer driver. Should be fine. But do check if there are any updates for it.

Other than that, have a great weekend,

- Ludek

Former Member
0 Kudos

Hi Ludek,

I hope you had a good weekend.

Well, I managed to run Modules64 as instructed and take screenshots of the detailed list and do the comparison. It took a while, as there are a total of over 400 DLLs associated with the application. The CR and DB DLLs are all the same version (13.0.1.220 for CR).

Another test we did was remove CR from our app completely and run this script and it ran several times longer than it has before and did not cause an APPCRASH.

Any ideas?

Thanks,

Zamir

Former Member
0 Kudos

I should add that there were some DLLs that appeared only in the DEV environment or only in the PROD environment. For the most part, these differences were expected or explainable. However, there was one CR-related difference. The crashed PROD machine had a "CRLOV.DLL" loaded, while the dev machine did not.

Also, because our DEV machines receive windows update and the PROD machine do not (I know, far from ideal), there are numerous version differences for windows DLLs. I don't know if any of them are relevant to CR, but I'll list a few that stood out to me, just in case:

(in each case below, the DEV machine has a newer version)

BCRYPTPRIMITIVES.DLL

COMCTL.DLL

DIASYMREADER.DLL

GDI32.DLL

GDIPLUS.DLL

IEFRAME.DLL

IERUTIL.DLL

IMAGEHELP.DLL

USP10.DLL (there were two versions of this DLL loaded on each machine. The CR version was the same on both, but the windows version differed).

Trying to provide as much information as I can

former_member183750
Active Contributor
0 Kudos

Hello Zamir:

The Win updates are bothersome...

USP10.dll would certainly be something to look at. Make sure that the one used by your app is from the CR bin directory(C:\Program Files (x86)\SAP BusinessObjects\Crystal Reports for .NET Framework 4.0\Common\SAP BusinessObjects Enterprise XI 4.0\win32_x86).

The other system dlls - who knows... If there is A prod machine that we can get updated to the level of the development machine, that would be great.

You don't mention if you checked the database client dlls. Are these consistent between the dev and prod servers?

One other file to pay attention to; any u2l.dll or cruf.dll that might be loading from a directory other than the CR bin should be eliminated - at least as a test.

- Ludek

Former Member
0 Kudos

Hi Ludek,

USP10.dll would certainly be something to look at. Make sure that the one used by your app is from the CR bin directory(C:\Program Files (x86)\SAP BusinessObjects\Crystal Reports for .NET Framework 4.0\Common\SAP BusinessObjects Enterprise XI 4.0\win32_x86).

If Modules64 listed both this .dll and the one in windows\system32, does that mean it's using both?

The other system dlls - who knows... If there is A prod machine that we can get updated to the level of the development machine, that would be great.

Unfortunately that is not a possibility. If it came to the point where we'd have to update our entire system image (in the field as well), we'd unfortunately be better served by removing our dependency on CR.

You don't mention if you checked the database client dlls. Are these consistent between the dev and prod servers?

When I said "DB", I meant database. They are consistent.

One other file to pay attention to; any u2l.dll or cruf.dll that might be loading from a directory other than the CR bin should be eliminated - at least as a test.

Only found U2L* dlls and they are all loaded from CR.

former_member183750
Active Contributor
0 Kudos

If Modules64 listed both this .dll and the one in windows\system32, does that mean it's using both?

- Yes. that means the app is using both - a potential issue that you may want to look at. Normally I would not be too concerned, but in your case, who knows.

Re.; Unfortunately that is not a possibility. If it came to the point where we'd have to update our entire system image (in the field as well), we'd unfortunately be better served by removing our dependency on CR.

- Umm, a tough message here... sorry. As I see it, we have an unpatched OS and an application is not working there as expected. On a patched OS, the app works as expected. The solution is to remove your dependency on Crystal Reports(?). Why does that not make any sense to me?

- Ludek

Former Member
0 Kudos

Umm, a tough message here... sorry. As I see it, we have an unpatched OS and an application is not working there as expected. On a patched OS, the app works as expected. The solution is to remove your dependency on Crystal Reports(?). Why does that not make any sense to me?

The windows updates are not the only differences between dev environments and production environments. If we were trying to do a true "diff" between the systems, it would be near impossible, since there are hardware differences as well.

From my point of view, updating Windows would be a revolving door. If we are acknowledging that CR relies on specific Windows updates to be stable, then who is to say that a new bug won't appear that will require another update, etc., ad nauseum. That is why, if the solution is to update, I would prefer to create a simpler, stabler alternative with fewer dependencies (e.g. not having to depend on a specific update from Windows).

I don't think that looking at the differences between dev and prod is a tractable approach.

What of the more direct line: what could cause a corrupt heap in Crystal Reports, perhaps in the CrystalReportsViewer?

former_member183750
Active Contributor
0 Kudos

Well Zamir, with all due respect, what you're saying is essentially this;

I have a car and never ever take it for a tune up. Worked like a charm for a while, now it sputters and belches black smoke. But rather than taking it for a tune up, I'll throw out the engine from Acme and put in a new one from DifferentAcme...(?).

Have a look at printjob limit (see [Crystal Reports Maximum Report Processing Jobs Limit|http://www.sdn.sap.com/irj/boc/index?rid=/library/uuid/f053713e-3e3d-2c10-2a81-f79259e54023]

Rather than just focusing on is the applicationu2019s error message. Is there anything different in the event logs?

See if you can rename the usp10.dll that is loading from (system32(?))

See if enabling the report option u201CNo Printeru201D and "Dissociate Formatting Page Size and Printer Paper Size" will help.

And that will be pretty well it. Other than that, unless we can do a tune up, it may be time to go to DifferentAcme.

- Ludek

Former Member
0 Kudos

I should add that another reason I don't feel it's useful to compare dev and prod environments is that, as I understand it, the heap while debugging is allocated differently than when not debugging. As this problem is likely due to a heap corruption occurring in CR, it is not surprising that they behave differently and it's something that a Windows update would have no bearing on.

Former Member
0 Kudos

Well Zamir, with all due respect, what you're saying is essentially this;

I have a car and never ever take it for a tune up. Worked like a charm for a while, now it sputters and belches black smoke. But rather than taking it for a tune up, I'll throw out the engine from Acme and put in a new one from DifferentAcme...(?).

Ludek, I understand your point of view as these kind of issues are very difficult to debug with unmatched environments, but I don't believe that analogy is a fair one. If I were to use the car analogy, this would be my take:

Instead of the engine sputtering and belching black smoke, it stops running about once in every 100 times that I the use cruise control feature (a dangerous problem!). Rather than focusing on the cruise control and it's interaction with the engine, the advice is to try it with a new engine and see if that works, with no guarantees that this will address the root cause of the problem. Even if the cruise control works after replacing the engine, the problem may just be dormant, waiting for a different usage pattern of the cruise control to awaken it.

Regardless, this is devolving and that is not my intent. I appreciate your further suggestions and I'm hoping we can still "drive" towards a solution

Have a look at printjob limit (see Crystal Reports Maximum Report Processing Jobs Limit

I read over the document, but I think we are safe here, as our single report does not contain so many sub-elements and report creation is serialized. Another developer has checked that we Dispose and Close appropriately.

Rather than just focusing on is the applicationu2019s error message. Is there anything different in the event logs?

I only see a log entry for the crash (which occurs directly after opening the form with the report). The only additional information I see in this message is the that the "faulting module" is identified as ntdll.dll, but this is always the "faulting" module for heap corruption, from what I've seen reported.

See if you can rename the usp10.dll that is loading from (system32(?))

This is worth a shot, I am trying it right now. Unfortunately, our last system ran for over 10 hours before crashing, so it takes a while to test potential fixes.

See if enabling the report option u201CNo Printeru201D and "Dissociate Formatting Page Size and Printer Paper Size" will help.

I will try this next. I'm curious about the rationale for this, as we do allow an option for the user to print in the report dialog.

And that will be pretty well it. Other than that, unless we can do a tune up, it may be time to go to DifferentAcme.

Hopefully it doesn't come to that

- Ludek

Former Member
0 Kudos

The machine running with a re-named USP10.DLL just suffered the same crash

Will try the "No Printer" suggestion.

Former Member
0 Kudos

Well, I think we found the problem, and I'm posting it for others who may run into this hard-to-debug failure.

As I had said in the initial post

"CR is not the only part of our application that uses unmanaged memory, but this crash consistently happens when opening the dialog with a CR component in it (which renders a report)."

This is why the investigation started with CR, even though we were fully aware that any of the other unmanaged components could be corrupting the heap. After checking about every possible angle on our CR deployment with Ludek's help, it seemed to be sound and I decided to start looking at the other components. We also use (or should I say "used"? ) some compiled Matlab in our application to do some heavy calculations. I found that when disabling this functionality, the crash would not occur - we found our culprit.

Needless to say, we are replacing that component.

Ludek, thank you for your help. I hope you can understand why we initially suspected CR, with the little information we had to go on. In the end, neither windows updates nor replacing CR would have solved the problem - the latter would have only masked it. For anyone that runs into this error, it will be wise to discuss with them any and all other components that use unmanaged memory.

Answers (0)