Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
238 views
in Technique[技术] by (71.8m points)

CSV file generation in every 2 hours using Java

I have a requirement in our project to generate a CSV file every 2 hours using a Java program. I want to create date wise folder under hourly based CSV file.It means every CSV file contains 2 hours(Configurable) data. I am able to create hourly basis CSV file. But i not able to create configurable hourly basis CSV.I am not using ScheduledExecutorService. I want simple way to solve this problem.

public static boolean writeFileHourlyBasis(String tenantName, String clusterName, String contentSourceName,
        String fileContent) {

    String reconciliationLogPath = AppConfigHashMap.getPropertyValue("ReconciliationLogFolder");

    if (reconciliationLogPath != null) {
        File fullReconciliationLogPath = new File(reconciliationLogPath + File.separator + tenantName
                + File.separator + clusterName + File.separator + contentSourceName + File.separator + formatter.format(new Date()));
        if (fullReconciliationLogPath.mkdirs() == true){

            logger.debug("Folder " + fullReconciliationLogPath + " created.");
        }
        try {
            SimpleDateFormat dateFormat = new SimpleDateFormat("HH");
            String fileName = tenantName + "_" + dateFormat.format(new Date()) + ".csv";
            File file1 = new File(fullReconciliationLogPath, fileName);
            String headerString = "";           

            if (!file1.exists()) {
                headerString = "DateTimeUTC,DateTime,Transcript->TranscriptId,Target->ReturnCode,Target->Status,Target->Message,MessageId,StatusMessage 
";
            }else{
                if(isNewLine){
                    headerString="
";
                }
                isNewLine=false;
            }
            FileWriter datawriter = new FileWriter(file1, true);
            datawriter.append(headerString);
            datawriter.append(fileContent);
            datawriter.append("
");
            datawriter.flush();
            datawriter.close();
        } catch (Exception e) {
            logger.warn("Exception occurred while writing Reconcilation log :" + e.getMessage());
            return false;
        }
    }else{
        //TODO: log.error("Reconciliation log enabled, but path not provided!");
        logger.warn("Reconciliation log enabled, but path not provided!");
        return false;
    }
    return true;

}

Can any one help me to solve this problem.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

I am not using ScheduledExecutorService. I want simple way to solve this problem.

You already have your answer: The Executors framework was added to Java to be that simple way to solve this problem. Executors abstract away the tricky messy details of handling background tasks and threading.

Your should be using a ScheduledExecutorService for your purpose.

Executors may seem daunting at first glance, but their use in practice is quite simple. Read the Oracle Tutorial. Then look at code examples in Stack Overflow and other blogs etc.

Tip: Search Stack Overflow using an external search engine such as DuckDuckGo/Bing/Google with the site:StackOverflow.com criterion. The built-in search feature in Stack Overflow is anemic and is biased towards Questions rather than Answers.

Define your executor service, backed by a thread pool.

ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();

Define your task to be run as a Runnable.

Runnable task = new Runnable() {
    @Override
    public void run () {
        … stuff to do goes here
    }
}

Pass the Runnable object to the executor service. Tell it how often to run. You also pass an initial delay; here we pass zero to begin immediately.

scheduledExecutorService.scheduleAtFixedRate( task , 0 , 2 , TimeUnit.HOURS ); 

Tip: Two critical things to know about using ScheduledExecutorService (SES):

  • Always gracefully shutdown the SES when no longer needed or when your app exits. Otherwise the threadpool may survive and continue working, like a zombie.
  • Always catch all Exceptions, and perhaps Errors, in the task being submitted to the SES. If any exception or error bubbles up to reach the SES, all further work by the SES creases. The stoppage is silent, no warning, no logs.

So modify the Runnable seen above.

Runnable task = new Runnable() {
    @Override
    public void run () {
        try {
            … stuff to do goes here
        } catch ( Exception e ) {
            … Handle any unexpected exceptions bubbling up to avoid silently killing your executor service. 
        }
    }
}

Tip: Never use the terrible old date-time classes such as Date, Calendar, SimpleDateFormat. They were supplanted years ago by the modern java.time classes.


Tip: When writing a moment in a file name, follow the ISO 8601 standard for representing date-time values as text.

These formats are used by default in the java.time classes when parsing or generating strings.

For file names you may want to use the alternative “basic” formats defined in the standard. Basic means the use of delimiters is minimized.

Be sure to avoid backward slash, forward slash, and colon characters. These are forbidden in DOS/Windows, Unix, and macOS/iOS file systems, respectively.


Tip: Don’t write your own CSV code. Use Apache Commons CSV library to read and write CSV or Tab-delimited files. It works well in my experience.


Example app

Here is an entire example contained in a single .java file.

We have a few objects of our little class Event. On each scheduled run, we update the timestamp on each Event object. Then we write all the member variables of all those Event objects to a text file in CSV format using the Apache Commons CSV library. Each run is defined by a Runnable object.

The runs are scheduled by a ScheduledExecutorService backed by a thread pool with a single thread.

In real work I would not squeeze all this into a single .java file. But doing so here makes for a nice compact demo.

package com.basilbourque.example;

import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;

import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.time.*;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.List;
import java.util.Locale;
import java.util.UUID;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

public class ExportToCsv {
    public static void main ( String[] args ) {
        ExportToCsv app = new ExportToCsv();
        app.doIt();
    }

    private void doIt () {
        System.out.println( "TRACE - doIt running at " + ZonedDateTime.now() );
        List< Event > events = List.of(
        new Event( UUID.randomUUID() , "alpha" , Instant.now() ) ,
        new Event( UUID.randomUUID() , "beta" , Instant.now() ) ,
        new Event( UUID.randomUUID() , "gamma" , Instant.now() )
        );

        Runnable task = new Runnable() {
            @Override
            public void run () {
                // Nest all this stuff of your `run` method into a `try-catch( Exception e )` to avoid having your executor cease silently.
                Instant start = Instant.now();
                System.out.print( "TRACE - Runnable starting at " + start + ". " );
                // Update the moment recorded in each `Event` object.
                events.forEach( ( event ) -> event.update() );
                // Export to CSV. Using “Apache Commons CSV” library. https://commons.apache.org/proper/commons-csv/
                // Get current moment in UTC. Lop off the seconds and fractional second. Generate text without delimiters.
                String dateTimeLabel = OffsetDateTime.now( ZoneOffset.UTC ).truncatedTo( ChronoUnit.MINUTES ).format( DateTimeFormatter.ofPattern( "uuuuMMdd'T'HHmmX" , Locale.US ) );
                String fileNamePath = "myCsv_" + dateTimeLabel + ".csv";
                try (  // Try-with-resources syntax automatically closes any passed objects implementing `AutoCloseable`, even if an exception is thrown.
                BufferedWriter writer = new BufferedWriter( new FileWriter( fileNamePath ) ) ;
                CSVPrinter csvPrinter = new CSVPrinter( writer , CSVFormat.DEFAULT.withHeader( "Id" , "Name" , "When" ) ) ;
                ) {
                    for ( Event event : events ) {
                        csvPrinter.printRecord( event.id , event.name , event.when );
                    }
                    csvPrinter.flush();
                } catch ( IOException e ) {
                    // TODO: Handle i/o exception when creating or writing to file in storage.
                    e.printStackTrace();
                }
                Instant stop = Instant.now() ;
                System.out.println( "Runnable ending its run at " + start + ". Duration: " + Duration.between( start , stop ) + ".");
            }
        };

        // Schedule this task. Currently set to run every two minutes, ending after 20 minutes. Adjust as desired.
        ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();  // Using a single thread here, as we have only a single series of tasks to be executed, no multi-tasking. 
        try {
            scheduledExecutorService.scheduleAtFixedRate( task , 0 , 2 , TimeUnit.MINUTES );  // Schedule our task to run every so often.
            try {
                Thread.sleep( TimeUnit.MINUTES.toMillis( 20 ) );  // Sleep this main thread for a while to let our task running on the background thread do its thing a few times.
            } catch ( InterruptedException e ) {
                System.out.println( "TRACE - Our main thread was woken earlier than expected, and interrupted our run. " );
                e.printStackTrace();
            }
        } finally {
            System.out.println( "Shutting down the scheduledExecutorService at " + ZonedDateTime.now() );  // Generally best to log in UTC, `Instant.now()`.
            scheduledExecutorService.shutdown();  // Always shutdown your executor, as it may otherwise survive your app exiting, becoming a zombie, continuing to run endlessly.
        }
        System.out.println( "App running on main thread ending at " + Instant.now() + "." );
    }

    class Event {
        public UUID id;
        public String name;
        public Instant when;

        public Event ( UUID id , String name , Instant when ) {
            this.id = id;
            this.name = name;
            this.when = when;
        }

        public void update () {
            this.when = Instant.now();
        }
    }
}

When run.

TRACE - doIt running at 2018-09-24T20:16:25.794081-07:00[America/Los_Angeles]

TRACE - Runnable starting at 2018-09-25T03:16:25.832025Z. Runnable ending its run at 2018-09-25T03:16:25.832025Z. Duration: PT0.025342S.

TRACE - Runnable starting at 2018-09-25T03:18:25.829634Z. Runnable ending its run at 2018-09-25T03:18:25.829634Z. Duration: PT0.001121S.

Files seen in this screenshot.

enter image description here


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

1.4m articles

1.4m replys

5 comments

57.0k users

...