Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
3.8k views
in Technique[技术] by (71.8m points)

java - Error while exporting table data from Hadoop to Mysql

While exporting the table data from hadoop to mysql using sqoop export command,i am facing this below error.Appropriate permissions and user ownership are there in place for the files and directories.

Sqoop command used:

 sqoop export --connect jdbc:mysql://localhost:3306/test --table retailinfo --username root --password abc --export-dir /user/root/ --fields-terminated-by ',' --lines-terminated-by '
'

Errors

21/01/16 11:45:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.15.1
21/01/16 11:45:27 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/01/16 11:45:27 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
21/01/16 11:45:27 INFO tool.CodeGenTool: Beginning code generation
21/01/16 11:45:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `retailinfo` AS t LIMIT 1
21/01/16 11:45:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `retailinfo` AS t LIMIT 1
21/01/16 11:45:27 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
Note: /tmp/sqoop-root/compile/776bbbaf6e0724cce128bb1fe8066127/retailinfo.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
21/01/16 11:45:29 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/776bbbaf6e0724cce128bb1fe8066127/retailinfo.jar
21/01/16 11:45:29 INFO mapreduce.ExportJobBase: Beginning export of retailinfo
21/01/16 11:45:29 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
21/01/16 11:45:30 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
21/01/16 11:45:30 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
21/01/16 11:45:30 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
21/01/16 11:45:30 INFO client.RMProxy: Connecting to ResourceManager at ip-172-31-60-27.ec2.internal/172.31.60.27:8032
21/01/16 11:45:32 INFO input.FileInputFormat: Total input paths to process : 2
21/01/16 11:45:32 INFO input.FileInputFormat: Total input paths to process : 2
21/01/16 11:45:32 INFO mapreduce.JobSubmitter: number of splits:3
21/01/16 11:45:32 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
21/01/16 11:45:32 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1610795706578_0002
21/01/16 11:45:32 INFO impl.YarnClientImpl: Submitted application application_1610795706578_0002
21/01/16 11:45:33 INFO mapreduce.Job: Running job: job_1610795706578_0002
21/01/16 11:45:37 INFO mapreduce.Job: Job job_1610795706578_0002 running in uber mode : false
21/01/16 11:45:37 INFO mapreduce.Job:  map 0% reduce 0%
21/01/16 11:45:37 INFO mapreduce.Job: Job job_1610795706578_0002 failed with state FAILED due to: **Application application_1610795706578_0002 failed 2 times due to AM Container for 

appattempt_1610795706578_0002_000002 exited with exitCode: 1**

Yarn error logs:

yarn logs -applicationId application_1610795706578_0002

LogType:stderr
Log Upload Time:Sat Jan 16 10:42:09 +0000 2021
LogLength:1447
Log Contents:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.service.CompositeService
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        ... 13 more

LogType:stdout
Log Upload Time:Sat Jan 16 10:42:09 +0000 2021
LogLength:0
Log Contents:


LogType:stderr
Log Upload Time:Sat Jan 16 10:42:09 +0000 2021
LogLength:1447
Log Contents:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.service.CompositeService
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        ... 13 more

LogType:stdout
Log Upload Time:Sat Jan 16 10:42:09 +0000 2021
LogLength:0
Log Contents:

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
等待大神解答

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...