Skip to Content

ESP (CSV) Output Adapter for rolling log-files

Hi all,

what we want:

We have an output stream and want to write it's contents to a file (csv or json). This file is used as kind of a log file for custom error messages and warnings.

In order to limit storage consumption we would like to limit the amount of files that can exist at the same time. Whenever the limit is it the oldest file shall be deleted.

we are currently using the File/Hadoop CSV as follows:

ATTACH OUTPUT ADAPTER csv_error_output

TYPE toolkit_file_csv_output

TO output_stream_error

PROPERTIES

dir = '<someFolderName>',

file = 'log',

timeBasedRotate = true,

timestampInFilenames = true,

timeBasedRotateInterval = 3600,

timestampInFilenamesFormat = 'HH',

;


Expected result would be files in the following format:

log.01 - for 1AM

log.02 - for 2AM

log.03 - ...

...

Where the number represents an hour. This would lead to overwriting the log files after 24 hours.

However ESP always adds a counting number to the files like so:

log.001.01

log.002.02

log.003.02

...

log.325.08

which should actually only happen if you set the fileSizeLimit - Property which we did not do (according to the documentation).

Is there an alternative way or a better solution?

Thanks,

Sven

Add a comment
10|10000 characters needed characters exceeded

Related questions

1 Answer

  • author's profile photo Former Member
    Former Member
    Posted on Apr 07, 2016 at 09:01 AM

    I have the same problem. If somebody know the resolution about it, please let me know.

    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.