您的足迹:首页 > Hadoop >Sqoop2安装以及使用

Sqoop2安装以及使用

sqoop2安装部署

我使用的是在线安装,由于sqoop2要比sqoop1更优秀,这里不再赘述sqoop1的安装。

安装Sqoop2

wget http://apache.dataguru.cn/sqoop/1.99.6/sqoop-1.99.6-bin-hadoop200.tar.gz

tar -xzvf sqoop-1.99.6-bin-hadoop200.tar.gz

mv sqoop-1.99.6-bin-hadoop200 sqoop-1.99.6

cd sqoop-1.99.6

sudo apt-get install zip

bin/addtowar.sh -hadoop-version 2.2.0 -hadoop-path /home/hadoop-2.2.0

bin/addtowar.sh -jars /home/mysql-connector-java-5.1.18.jar

vi server/conf/sqoop.properties

修改org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/etc/hadoop/conf/为

org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/home/hadoop-2.2.0/conf/

启动Sqoop 2 server:

bin/sqoop.sh server start

http://host001:12000/sqoop/

停止Sqoop 2 server:

bin/sqoop.sh server stop

客户端连接Sqoop 2 server:

客户端直接解压即可运行

MySQL准备数据库和表:

create database test;

create table history (userId int, command varchar(20));

insert into history values(1, 'ls');

insert into history values(1, 'dir');

insert into history values(2, 'cat');

insert into history values(5, 'vi');

交互模式:

bin/sqoop.sh client

sqoop:000> set server --host host001 --port 12000 --webapp sqoop

sqoop:000> show version --all

sqoop:000> show connector --all

sqoop:000>create connection --cid 1

Name: mysql

JDBC Driver Class: com.mysql.jdbc.Driver

JDBC Connection String:

jdbc:mysql://host001:3306/test?useUnicode=true&characterEncoding=UTF-8&createDatabaseIfNotExist=true&autoReconnect=true

Username: root

Password: ***

entry#回车

Max connections:100

sqoop:000>create job --xid 1 --type import

Name:ImportHistory

Schema name:

Table name: history

Table SQL statement:

Table column names:

Partition column name:userId

Boundary query:

Choose:0

Choose: 0

Output directory: output-sqoop2-history

Extractors:

Loaders:

sqoop:000> submission start --jid 1

sqoop:000> submission status --jid 1

sqoop:000> submission stop --jid 1

批处理模式:

sqoop.sh client /home/script.sqoop

vi /home/script.sqoop

输入:

     #指定服务器信息

     set server --host host001 --port 12000 --webapp sqoop

     #执行JOB

     submission start --jid 1

本博客所有文章如无特别注明均为原创。作者:数据为王复制或转载请以超链接形式注明转自 数据为王
原文地址《Sqoop2安装以及使用

相关推荐


  • blogger

发表评论

路人甲 表情
看不清楚?点图切换 Ctrl+Enter快速提交

网友评论(0)