key获取json的key和valueemaxsize最大多少
点击联系发帖人
时间：2017-01-21 05:41
博客访问： 490169
博文数量： 167
注册时间：
分类： Hadoop 16:58:02
5.在安装的时候遇到的问题
5.1使用ambari-server start的时候出现ERROR: Exiting with exit code -1.
5.1.1REASON: Ambari Server java process died with exitcode
255. Check /var/log/ambari-server/ambari-server.out for more information
由于是重新安装，所以在使用/etc/init.d/postgresql& initdb初始化数据库的时候会出现这个错误，所以需要
先用yum –y remove postgresql*命令把postgresql卸载
然后把/var/lib/pgsql/data目录下的文件全部删除
然后再配置postgresql数据库（执行1.6章节内容）
然后再次安装（3章节内容）
5.1.2在日志中有如下错误：ERROR [main] AmbariServer:820 - Failed to run
the Ambari Server
com.google.inject.ProvisionException: Guice
provision errors:
1) Error injecting method,
java.lang.NullPointerException
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:243)
org.apache.ambari.server.api.services.AmbariMetaInfo.class(AmbariMetaInfo.java:125)
while locating org.apache.ambari.server.api.services.AmbariMetaInfo
for field at
org.apache.ambari.server.controller.AmbariServer.ambariMetaInfo(AmbariServer.java:145)
org.apache.ambari.server.controller.AmbariServer.class(AmbariServer.java:145)
while locating org.apache.ambari.server.controller.AmbariServer
at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:813)
Caused by: java.lang.NullPointerException
org.apache.ambari.server.stack.StackModule.processRepositories(StackModule.java:665)
org.apache.ambari.server.stack.StackModule.resolve(StackModule.java:158)
at org.apache.ambari.server.stack.StackManager.fullyResolveStacks(StackManager.java:201)
org.apache.ambari.server.stack.StackManager.(StackManager.java:119)
org.apache.ambari.server.stack.StackManager$$FastClassByGuice$$33e4ffe0.newInstance()
&&&&&&& at
com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)
at com.sun.proxy.$Proxy26.create(Unknown Source)
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:247)
5.2安装HDFS和HBASE的时候出现/usr/hdp/current/hadoop-client/conf &doesn't exist
5.2.1/etc/Hadoop/conf文件链接存在
是由于/etc/hadoop/conf和/usr/hdp/current/hadoop-client/conf目录互相链接，造成死循环，所以要改变一个的链接
cd /etc/hadoop
rm -rf conf
ln -s /etc/hadoop/conf.backup
/etc/hadoop/conf
HBASE也会遇到同样的问题，解决方式同上
cd /etc/hbase
rm -rf conf
ln -s /etc/hbase/conf.backup /etc/hbase/conf
ZooKeeper也会遇到同样的问题，解决方式同上
cd /etc/zookeeper
rm -rf conf
ln -s /etc/zookeeper/conf.backup
/etc/zookeeper/conf
5.2.2/etc/Hadoop/conf文件链接不存在
查看正确的配置，发现缺少两个目录文件config.backup和2.4.0.0-169，把文件夹拷贝到/etc/hadoop目录下
重新创建/etc/hadoop目录下的conf链接：
/etc/hadoop
rm -rf conf
/usr/hdp/current/hadoop-client/conf conf
5.3在认证机器(Confirm
Hosts)的时候出现错误Ambari agent machine hostname (localhost) does not match expected
ambari server hostname
Ambari配置时在Confirm Hosts的步骤时，中间遇到一个很奇怪的问题：总是报错误：
Ambari agent machine hostname
(localhost.localdomain) does not match expected ambari server hostname (xxx).
后来修改的/etc/hosts文件中
127.0.0.1&& localhost dsj-kj1
::1&&&&&&&& localhost dsj-kj1
10.13.39.32&&&& dsj-kj1
10.13.39.33&&&& dsj-kj2
10.13.39.34&&&& dsj-kj3
10.13.39.35 &&&&dsj-kj4
10.13.39.36&&&& dsj-kj5
127.0.0.1& localhost
localhost.localdomain localhost4 localhost4.localdomain4
::1&&&&&&&& &localhost localhost.localdomain localhost6
localhost6.localdomain6
10.13.39.32&&&& dsj-kj1
10.13.39.33&&&& dsj-kj2
10.13.39.34&&&& dsj-kj3
10.13.39.35&&&& dsj-kj4
10.13.39.36&&&& dsj-kj5
感觉应该是走的ipv6协议，很奇怪，不过修改后就可以了。
5.4ambary-server重装
删除使用脚本删除
注意删除后要安装两个系统组件
yum -y install ruby*
yum -y install redhat-lsb*
yum -y install snappy*
5.5Ambari连接mysql设置
在主节点把mysql数据库连接包拷贝在/var/lib/ambary-server/resources目录下并改名为mysql-jdbc-driver.jar
/usr/share/java/mysql-connector-java-5.1.17.jar
/var/lib/ambari-server/resources/mysql-jdbc-driver.jar
再在图形界面下启动hive
5.6在注册机器(Confirm Hosts)的时候出现错误Failed
to start ping port listener of: [Errno 98] Address already in use
某个端口或者进程一直陪占用
解决方法：
发现df命令一直执行没有完成，
[root@testserver1 ~]# netstat -lanp|grep 8670
tcp&&&&&&& 0&&&&& 0
0.0.0.0:8670&&&&&&&&&&&&&&&
0.0.0.0:*&&&&&&&&&&&&&&&&&&
LISTEN&&&&& 2587/df
[root@testserver1 ~]# kill -9 2587
kill后，再重启ambari-agent问题解决
[root@testserver1 ~]# service ambari-agent restart
Verifying Python version compatibility...
Using python& /usr/bin/python2.6
ambari-agent is not running. No PID found at
/var/run/ambari-agent/ambari-agent.pid
Verifying Python version compatibility...
Using python& /usr/bin/python2.6
Checking for previously running Ambari Agent...
Starting ambari-agent
Verifying ambari-agent process status...
Ambari Agent successfully started
Agent PID at: /var/run/ambari-agent/ambari-agent.pid
Agent out at: /var/log/ambari-agent/ambari-agent.out
Agent log at: /var/log/ambari-agent/ambari-agent.log
5.7在注册机器(Confirm Hosts)的时候出现错误The following hosts have Transparent HugePages (THP) enabled。THP should
be disabled to avoid potential Hadoop performance issues
解决方法：
在Linux下执行：
echo never
&/sys/kernel/mm/redhat_transparent_hugepage/defrag
echo never &/sys/kernel/mm/redhat_transparent_hugepage/enabled
echo never &/sys/kernel/mm/transparent_hugepage/enabled
echo never &/sys/kernel/mm/transparent_hugepage/defrag
5.8启动hive的时候出现错误unicodedecodeerror ambari in position 117
查看/etc/sysconfig/i18n文件，发现内容如下：
LANG=”zh_CN.UTF8”
原来系统字符集设置成了中文，改成如下内容，问题解决：
LANG="en_US.UTF-8"
5.9安装Metrics的时候报如下错误，安装包找不到
1.failure: Updates-ambari-2.2.1.0/ambari/ambari-metrics-monitor-2.2.1.0-161.x86_64.rpm
from HDP-UTILS-1.1.0.20: [Errno 256] No more mirrors to try.
在ftp源服务器上执行命令：
cd /var/www/html/ambari/HDP-UTILS-1.1.0.20/repos/centos6
mkdir Updates-ambari-2.2.1.0
cp -r /var/www/html/ambari/Updates-ambari-2.2.1.0/ambari
/var/www/html/ambari/HDP-UTILS-1.1.0.20/repos/centos6/Updates-ambari-2.2.1.0
然后重新生成repodata
cd /var/www/html/ambari
rm -rf repodata
createrepo ./
2.failure: HDP-UTILS-1.1.0.20/repos/centos6/Updates-ambari-2.2.1.0/ambari/ambari-metrics-monitor-2.2.1.0-161.x86_64.rpm
from HDP-UTILS-1.1.0.20: [Errno 256] No more mirrors to try.
在/etc/yum.repos.d目录下删除mnt.repo，并使用yum clean all命令来清空yum的缓存
cd /ec/yum.repos.d
rm -rf mnt.repo
yum clean all
5.11jps 报process information unavailable解决办法
4791 -- process information unavailable
解决办法：
进入tmp目录，
删除该目录下
名称为hsperfdata_{ username}的文件夹
然后jps,清净了。
ls -l | grep hsperf | xargs rm
ls -l | grep hsperf
5.12namenode启动报错在日志文件中ERROR
namenode.NameNode (NameNode.java:main(1712)) - Failed to start namenode
日志中还有java.net.BindException: Port in use: gmaster:50070
Caused by: java.net.BindException: Address
already in use
判断原因是50070上一次没有释放，端口占用
netstat下time_wait状态的tcp连接：&
1.这是一种处于连接完全关闭状态前的状态；&
2.通常要等上4分钟（windows server）的时间才能完全关闭；&
3.这种状态下的tcp连接占用句柄与端口等资源，服务器也要为维护这些连接状态消耗资源；&
4.解决这种time_wait的tcp连接只有让服务器能够快速回收和重用那些TIME_WAIT的资源：修改注册表[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters]添加dword值TcpTimedWaitDelay=30（30也为微软建议值；默认为2分钟）和MaxUserPort：65534(可选值5000 - 65534)；&
5.具体tcpip连接参数配置还可参照这里：http://technet.microsoft.com/zh-tw/library/ccv=ws.10%29.aspx&
6.linux下：&
vi /etc/sysctl.conf&
新增如下内容：&
net.ipv4.tcp_tw_reuse
net.ipv4.tcp_tw_recycle = 1&
net.ipv4.tcp_syncookies=1&
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_keepalive_time=1800
net.ipv4.tcp_max_syn_backlog=8192
使内核参数生效：&
[root@web02 ~]# sysctl -p&
net.ipv4.tcp_syncookies=1 打开TIME-WAIT套接字重用功能，对于存在大量连接的Web服务器非常有效。&
net.ipv4.tcp_tw_recyle=1&
net.ipv4.tcp_tw_reuse=1 减少处于FIN-WAIT-2连接状态的时间，使系统可以处理更多的连接。&
net.ipv4.tcp_fin_timeout=30 减少TCP KeepAlive连接侦测的时间，使系统可以处理更多的连接。&
net.ipv4.tcp_keepalive_time=1800 增加TCP SYN队列长度，使系统可以处理更多的并发连接。&
net.ipv4.tcp_max_syn_backlog=8192
5.13在启动的时候报错误resource_management.core.exceptions.Fail:
Execution of 'ambari-sudo.sh& -H -E /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh
在日志中有如下内容：
13:55:28,090 INFO&
security.ShellBasedIdMapping
(ShellBasedIdMapping.java:updateStaticMapping(322)) - Not doing static UID/GID
mapping because '/etc/nfs.map' does not exist.
13:55:28,096 INFO&
nfs3.WriteManager (WriteManager.java:(92)) - Stream timeout
is 600000ms.
13:55:28,096 INFO&
nfs3.WriteManager (WriteManager.java:(100)) - Maximum open
streams is 256
13:55:28,096 INFO&
nfs3.OpenFileCtxCache (OpenFileCtxCache.java:(54)) - Maximum
open streams is 256
13:55:28,259 INFO&
nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:(205)) - Configured
HDFS superuser is
13:55:28,261 INFO&
nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:clearDirectory(231)) - Delete
current dump directory /tmp/.hdfs-nfs
13:55:28,269 WARN& fs.FileUtil
(FileUtil.java:deleteImpl(187)) - Failed to delete file or dir
[/tmp/.hdfs-nfs]: it still exists.
说明hdfs这个用户对/tmp没有权限
赋予权限给hdfs用户：
chown &hdfs:hadoop /tmp
再启动问题解决
5.14在安装ranger组件的时候出现错误连接不上mysql数据库rangeradmin用户和不能赋权的问题
在数据库中先删除所有rangeradmin用户，注意使用drop user命令：
drop user 'rangeradmin'@'%';
drop user 'rangeradmin'@'localhost';
drop user 'rangeradmin'@'gmaster';
drop user 'rangeradmin'@'gslave1';
drop user 'rangeradmin'@'gslave2';
FLUSH PRIVILEGES;
再创建用户（注意gmaster是ranger安装的服务器机器名）
CREATE USER 'rangeradmin'@'%' IDENTIFIED BY
'rangeradmin';
GRANT ALL PRIVILEGES ON *.* TO
'rangeradmin'@'%'& with grant option;
CREATE USER 'rangeradmin'@'localhost'
IDENTIFIED BY 'rangeradmin';
GRANT ALL PRIVILEGES ON *.* TO
'rangeradmin'@'localhost'& with grant option;
CREATE USER 'rangeradmin'@'gmaster'
IDENTIFIED BY 'rangeradmin';
GRANT ALL PRIVILEGES ON *.* TO
'rangeradmin'@'gmaster'& with grant option;
FLUSH PRIVILEGES;
再查看权限：
SELECT DISTINCT CONCAT('User:
''',user,'''@''',host,''';') AS query FROM mysql.user
select * from mysql.user where
user='rangeradmin' \G;
5.15在ambari启动的时候出现错误：AmbariServer:820
- Failed to run the Ambari Server
这个问题困扰了我很久，最后通过查看源码找到了问题所在：
在/var/log/ambari-server/ambary-server.log文件中报有错误：
13 Apr :01,723& INFO [main] StackDirectory:458 - Stack
'/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS' doesn't contain an
upgrade directory
13 Apr :01,723& INFO [main] StackDirectory:468 - Stack
'/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS' doesn't contain
config upgrade pack file
13 Apr :01,744& INFO [main] StackDirectory:484 - Role command
order info was loaded from file:
/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS/role_command_order.json
13 Apr :01,840& INFO [main] StackDirectory:484 - Role command
order info was loaded from file:
/var/lib/ambari-server/resources/stacks/HDP/2.4/role_command_order.json
13 Apr :01,927 ERROR [main]
AmbariServer:820 - Failed to run the Ambari Server
com.google.inject.ProvisionException: Guice
provision errors:
1) Error injecting method,
java.lang.NullPointerException
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:243)
org.apache.ambari.server.api.services.AmbariMetaInfo.class(AmbariMetaInfo.java:125)
while locating org.apache.ambari.server.api.services.AmbariMetaInfo
for field at
org.apache.ambari.server.controller.AmbariServer.ambariMetaInfo(AmbariServer.java:145)
org.apache.ambari.server.controller.AmbariServer.class(AmbariServer.java:145)
while locating org.apache.ambari.server.controller.AmbariServer
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
&&&&&&&& at
org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:813)
Caused by: java.lang.NullPointerException
&&&&&&&& at
org.apache.ambari.server.stack.StackModule.processRepositories(StackModule.java:665)
&&&&&&&& at
org.apache.ambari.server.stack.StackModule.resolve(StackModule.java:158)
&&&&&&&& at
org.apache.ambari.server.stack.StackManager.fullyResolveStacks(StackManager.java:201)
&&&&&&&& at
org.apache.ambari.server.stack.StackManager.(StackManager.java:119)
&&&&&&&& at
org.apache.ambari.server.stack.StackManager$$FastClassByGuice$$33e4ffe0.newInstance()
&&&&&&&& at
com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
&&&&&&&& at
com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
&&&&&&&& at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
&&&&&&&& at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
&&&&&&&& at
com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)
&&&&&&&& at
com.sun.proxy.$Proxy26.create(Unknown Source)
&&&&&&&& at
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:247)
&&&&&&&& at
org.apache.ambari.server.api.services.AmbariMetaInfo$$FastClassByGuice$$202844bc.invoke()
&&&&&&&& at
com.google.inject.internal.cglib.reflect.$FastMethod.invoke(FastMethod.java:53)
&&&&&&&& at
com.google.inject.internal.SingleMethodInjector$1.invoke(SingleMethodInjector.java:56)
&&&&&&&& at
com.google.inject.internal.SingleMethodInjector.inject(SingleMethodInjector.java:90)
&&&&&&&& at
com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
&&&&&&&& at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
&&&&&&&& at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
&&&&&&&& at
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
&&&&&&&& at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
&&&&&&&& at
com.google.inject.Scopes$1$1.get(Scopes.java:65)
&&&&&&&& at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
&&&&&&&& at
com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
&&&&&&&& at
com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
&&&&&&&& at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
&&&&&&&& at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
&&&&&&&& at
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
&&&&&&&& at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
&&&&&&&& at
com.google.inject.Scopes$1$1.get(Scopes.java:65)
&&&&&&&& at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
&&&&&&&& ...
解决方法：
后来发现在文件/var/lib/ambari-server/resources/stacks/HDP/2.4/repos/repoinfo.xml中的内容os这行原来的如下：
5.16在启动hive的时候出现错误Error:
Duplicate key name 'PCS_STATS_IDX' (state=42000,code=1061)
在HIVE安装的机器cd /var/lib/ambari-agent/data目录下有日志，日志中报如下错误：
Traceback (most recent call last):
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 245, in
HiveMetastore().execute()
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 219, in execute
method(env)
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 58, in start
self.configure(env)
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 72, in configure
hive(name = 'metastore')
"/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py",
line 89, in thunk
return fn(*args, **kwargs)
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py",
line 292, in hive
user = params.hive_user
"/usr/lib/python2.6/site-packages/resource_management/core/base.py",
line 154, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 158, in run
self.run_action(resource, action)
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 121, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
line 238, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 70, in inner
result = function(command, **kwargs)
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 92, in checked_call
tries=tries, try_sleep=try_sleep)
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail:
Execution of 'export
HIVE_CONF_DIR=/usr/hdp/current/hive-metastore/conf/conf.
/usr/hdp/current/hive-metastore/bin/schematool -initSchema -dbType mysql
-userName hive -passWord [PROTECTED]' returned 1. WARNING: Use "yarn
jar" to launch YARN applications.
connection URL:&&&&&&&
jdbc:mysql://a2slave1/hive?createDatabaseIfNotExist=true
Connection Driver :&&&
com.mysql.jdbc.Driver
connection User:&&&&&& hive
metastore schema initialization to 1.2.1000
Initialization
script hive-schema-1.2.1000.mysql.sql
Duplicate key name 'PCS_STATS_IDX' (state=42000,code=1061)
org.apache.hadoop.hive.metastore.HiveMetaException:
Schema initialization FAILED! Metastore state would be inconsistent !!
schemaTool failed ***
解决方法：
HIVE安装的机器拷贝脚本hive-schema-1.2.1000.mysql.sql到：
[root@a2master /]# scp /usr/hdp/2.4.0.0-169/hive/scripts/metastore/upgrade/mysql/hive-schema-1.2.1000.mysql.sql root@a2slave1:/usr/local/mysql
hive-schema-1.2.1000.mysql.sql&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
100%&& 34KB& 34.4KB/s&&
在HIVE安装机器使用hive用户登陆，进入hive数据库，执行这个脚本：
[root@a2slave1 conf.server]# mysql -uhive
Enter password:
Welcome to the MySQL monitor.& C or \g.
Your MySQL connection id is 505
Server version: 5.6.26-log MySQL Community
Server (GPL)
Copyright (c) , Oracle and/or its
affiliates. All rights reserved.
Oracle is a registered trademark of Oracle
Corporation and/or its
affiliates. Other names may be trademarks
of their respective
Type '' or '\h' for help. Type '\c' to
clear the current input statement.
Reading table information for completion of
table and column names
You can turn off this feature to get a
quicker startup with -A
Database changed
mysql& source
/usr/local/mysql/hive-schema-1.2.1000.mysql.
5.17在启动hive的时候出现错误Could
not create "increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.
在/var/log/hive中的日志文件hiveserver.log中记录有：
10:45:20,446 INFO& [main]: server.HiveServer2
(HiveServer2.java:startHiveServer2(405)) - Starting HiveServer2
10:45:20,573 INFO& [main]: metastore.ObjectStore
(ObjectStore.java:initialize(294)) - ObjectStore, initialize called
10:45:20,585 INFO& [main]: metastore.MetaStoreDirectSql
(MetaStoreDirectSql.java:(140)) - Using direct SQL, underlying DB
10:45:20,585 INFO& [main]: metastore.ObjectStore (ObjectStore.java:setConf(277))
- Initialized ObjectStore
10:45:20,590 WARN& [main]: metastore.ObjectStore
(ObjectStore.java:getDatabase(577)) - Failed to get database default, returning
NoSuchObjectException
10:45:20,591 ERROR [main]: bonecp.ConnectionHandle
(ConnectionHandle.java:markPossiblyBroken(388)) - Database access problem.
Killing off this connection and all remaining connections in the connection
pool. SQL State = HY000
10:45:20,600 WARN& [main]: metastore.HiveMetaStore
(HiveMetaStore.java:createDefaultDB(623)) - Retrying creating default database
after error: Could not create "increment"/"table"
value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow
javax.jdo.JDOUserException: Could not create
"increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.
&&&&&&&& at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:549)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
&&&&&&&& at
org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)
&&&&&&&& at
sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
&&&&&&&& at
com.sun.proxy.$Proxy6.createDatabase(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:621)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
&&&&&&&& at
sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)
&&&&&&&& at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.init(CLIService.java:112)
&&&&&&&& at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.util.RunJar.run(RunJar.java:221)
&&&&&&&& at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)
NestedThrowablesStackTrace:
Could not create
"increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.
org.datanucleus.exceptions.NucleusUserException:
Could not create "increment"/"table" value-generation
container `SEQUENCE_TABLE` since autoCreate flags do not allow it.
&&&&&&&& at
org.datanucleus.store.rdbms.valuegenerator.TableGenerator.createRepository(TableGenerator.java:261)
&&&&&&&& at
org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obtainGenerationBlock(AbstractRDBMSGenerator.java:162)
&&&&&&&& at
org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerationBlock(AbstractGenerator.java:197)
&&&&&&&& at
org.datanucleus.store.valuegenerator.AbstractGenerator.next(AbstractGenerator.java:105)
&&&&&&&& at
org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueForGenerator(RDBMSStoreManager.java:2005)
&&&&&&&& at
org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1386)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3827)
&&&&&&&& at
org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManager.java:2571)
&&&&&&&& at
org.datanucleus.state.JDOStateManager.initialiseForPersistentNew(JDOStateManager.java:513)
&&&&&&&& at
org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:232)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.newObjectProviderForPersistentNew(ExecutionContextImpl.java:1414)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2218)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913)
&&&&&&&& at
org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
&&&&&&&& at
org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)
&&&&&&&& at
sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
&&&&&&&& at
com.sun.proxy.$Proxy6.createDatabase(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:621)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
&&&&&&&& at
sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)
&&&&&&&& at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.init(CLIService.java:112)
&&&&&&&& at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.util.RunJar.run(RunJar.java:221)
&&&&&&&& at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)
10:45:20,607 WARN& [main]: metastore.ObjectStore
(ObjectStore.java:getDatabase(577)) - Failed to get database default, returning
NoSuchObjectException
10:45:20,609 ERROR [main]:
bonecp.ConnectionHandle (ConnectionHandle.java:markPossiblyBroken(388)) -
Database access problem. Killing off this connection and all remaining
connections in the connection pool. SQL State = HY000
10:45:20,617 INFO& [main]: server.HiveServer2
(HiveServer2.java:stop(371)) - Shutting down HiveServer2
10:45:20,618 WARN& [main]: server.HiveServer2
(HiveServer2.java:startHiveServer2(442)) - Error starting HiveServer2 on
attempt 29, will retry in 60 seconds
java.lang.RuntimeException: Error applying
authorization policy on hive configuration: java.lang.RuntimeException: Unable
to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
&&&&&&&& at
org.apache.hive.service.cli.CLIService.init(CLIService.java:114)
&&&&&&&& at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.util.RunJar.run(RunJar.java:221)
&&&&&&&& at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException:
java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
&&&&&&&& at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:494)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.init(CLIService.java:112)
&&&&&&&& ...
Caused by: java.lang.RuntimeException:
Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
&&&&&&&& at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1533)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)
&&&&&&&& at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)
&&&&&&&& ...
Caused by:
java.lang.reflect.InvocationTargetException
&&&&&&&& at
sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)
&&&&&&&& ...
Caused by: javax.jdo.JDOUserException:
Could not create "increment"/"table" value-generation
container `SEQUENCE_TABLE` since autoCreate flags do not allow it.
NestedThrowables:
org.datanucleus.exceptions.NucleusUserException:
Could not create "increment"/"table" value-generation
container `SEQUENCE_TABLE` since autoCreate flags do not allow it.
&&&&&&&& at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:549)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
&&&&&&&& at
org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)
&&&&&&&& at
sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
&&&&&&&& at
com.sun.proxy.$Proxy6.createDatabase(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:625)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
&&&&&&&& ...
Caused by:
org.datanucleus.exceptions.NucleusUserException: Could not create
"increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.
&&&&&&&& at
org.datanucleus.store.rdbms.valuegenerator.TableGenerator.createRepository(TableGenerator.java:261)
&&&&&&&& at
org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obtainGenerationBlock(AbstractRDBMSGenerator.java:162)
&&&&&&&& at
org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerationBlock(AbstractGenerator.java:197)
&&&&&&&& at
org.datanucleus.store.valuegenerator.AbstractGenerator.next(AbstractGenerator.java:105)
&&&&&&&& at
org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueForGenerator(RDBMSStoreManager.java:2005)
&&&&&&&& at
org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1386)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3827)
&&&&&&&& at
org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManager.java:2571)
&&&&&&&& at
org.datanucleus.state.JDOStateManager.initialiseForPersistentNew(JDOStateManager.java:513)
&&&&&&&& at
org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:232)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.newObjectProviderForPersistentNew(ExecutionContextImpl.java:1414)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2218)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913)
&&&&&&&& at
org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727)
&&&&&&&& ...
最后发现是mysql数据库的binlog_format参数设置不正确，
原来设置的是STATEMENT，修改为MIXED
修改方法，在/etc/my.cnf文件中加上binlog_format=MIXED
然后重启mysql数据库
5.18在使用hive命令进入hive的时候报错误Permission
denied: user=root, access=WRITE,
inode="/user/root":hdfs:hdfs:drwxr-xr-x& hive
解决方法：
1.&&&&&& 使用HDFS的命令行接口修改相应目录的权限，hadoop fs -chmod 777 /user,后面的/user是要上传文件的路径，不同的情况可能不一样，比如要上传的文件路径为hdfs://namenode/user/xxx.doc，则这样的修改可以，如果要上传的文件路径为hdfs://namenode/java/xxx.doc，则要修改的为hadoop
fs -chmod 777 /java或者hadoop fs -chmod 777 /，java的那个需要先在HDFS里面建立Java目录，后面的这个是为根目录调整权限。
hadoop fs -chmod 777 /user
2.&&&&&& 在/etc/profile文件中加上系统的环境变量或java JVM变量里面添加export HADOOP_USER_NAME=hdfs(ambari使用的hadoop用户是hdfs)，这个值具体等于多少看自己的情况，以后会运行HADOOP上的Linux的用户名。
export HADOOP_USER_NAME=hdfs
5.19Spark Thrift
Server组件启动后自动关闭
Spark Thrift Server组件在启动后自动关闭，查看/var/log/spark下的日志文件spark-hive-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-a2master.out，中有如下内容：
10:26:10 WARN DomainSocketFactory: The short-circuit local reads feature cannot
be used because libhadoop cannot be loaded.
10:26:10 INFO Client: Requesting a new application from cluster with 3
NodeManagers
10:26:10 INFO Client: Verifying our application has not requested more than the
maximum memory capability of the cluster (512 MB per container)
10:26:10 ERROR SparkContext: Error initializing SparkContext.
java.lang.IllegalArgumentException: Required executor memory ( MB) is
above the max threshold (512 MB) of this cluster! Please check the values of
'yarn.scheduler.maximum-allocation-mb' and/or
'yarn.nodemanager.resource.memory-mb'.
&&&&&&& at
org.apache.spark.deploy.yarn.Client.verifyClusterResources(Client.scala:283)
&&&&&&& at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:139)
&&&&&&& at
org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
&&&&&&& at
org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
&&& &&&&at
org.apache.spark.SparkContext.(SparkContext.scala:530)
&&&&&&& at
org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:56)
&&&&&&& at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:76)
&&&&&&& at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
16/04/18 10:26:11 INFO ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages/stage/kill,null}
分析：原因是spark的内存设置的太小
解决方法：在前台界面修改内存为1536M，并把配置在其它机器上更新，重启spark服务
5.20hbase在启动的时候出现错误Permission
denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,0860883":hdfs:hdfs:drwxr-xr-x
在日志中有如下内容：
15:42:11,640 INFO& [regionserver/gslave2/192.168.1.253:16020]
hfile.CacheConfig: Allocating LruBlockCache size=401.60 MB, blockSize=64 KB
15:42:11,648 INFO& [regionserver/gslave2/192.168.1.253:16020]
hfile.CacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=433080,
freeSize=, maxSize=, heapSize=433080, minSize=,
minFactor=0.95, multiSize=, multiFactor=0.5, singleSize=,
singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false,
cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false,
cacheDataCompressed=false, prefetchOnOpen=false
15:42:11,704 INFO& [regionserver/gslave2/192.168.1.253:16020]
wal.WALFactory: Instantiating WALProvider of type class
org.apache.hadoop.hbase.wal.DefaultWALProvider
15:42:11,729 INFO& [regionserver/gslave2/192.168.1.253:16020]
regionserver.HRegionServer: STOPPED: Failed initialization
15:42:11,729 ERROR
[regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: Failed
org.apache.hadoop.security.AccessControlException:
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
&&&&&&&& at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
&&&&&&&& at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
&&&&&&&& at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2589)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2558)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:820)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:816)
&&&&&&&& at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:816)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:809)
&&&&&&&& at
org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:488)
&&&&&&&& at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
&&&&&&&& at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
&&&&&&&& at
org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:179)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1624)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1362)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:899)
&&&&&&&& at
java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
&&&&&&&& at
org.apache.hadoop.ipc.Client.call(Client.java:1411)
&&&&&&&& at
org.apache.hadoop.ipc.Client.call(Client.java:1364)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
&&&&&&&& at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
&&&&&&&& at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
&&&&&&&& at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:508)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)
&&&&&&&& at
com.sun.proxy.$Proxy16.mkdirs(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2587)
&&&&&&&& ...
15:42:11,732 FATAL
[regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: ABORTING
region server gslave2,8130424: Unhandled: Permission denied:
user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
org.apache.hadoop.security.AccessControlException:
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
&&&&&&&& at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
&&&&&&&& at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
&&&&&&&& at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2589)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2558)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:820)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:816)
&&&&&&&& at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:816)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:809)
&&&&&&&& at
org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:488)
&&&&&&&& at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
&&&&&&&& at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
&&&&&&&& at
org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:179)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1624)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1362)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:899)
&&&&&&&& at
java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
&&&&&&&& at
org.apache.hadoop.ipc.Client.call(Client.java:1411)
&&&&&&&& at
org.apache.hadoop.ipc.Client.call(Client.java:1364)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
&&&&&&&& at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
&&&&&&&& at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
&&&&&&&& at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:508)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)
&&&&&&&& at
com.sun.proxy.$Proxy16.mkdirs(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2587)
&&&&&&&& ...
15:42:11,732 FATAL
[regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer:
RegionServer abort: loaded coprocessors are: []
15:42:11,744 INFO& [regionserver/gslave2/192.168.1.253:16020]
regionserver.HRegionServer: Dump of metrics as JSON on abort: {
解决方法：
hdfs dfs -chown -R hbase:hbase /apps/hbase
阅读(17472) | 评论(0) | 转发(0) |
相关热门文章
给主人留下些什么吧！~~
请登录后评论。}
博文数量： 167
注册时间：
分类： Hadoop 16:58:02
5.在安装的时候遇到的问题
5.1使用ambari-server start的时候出现ERROR: Exiting with exit code -1.
5.1.1REASON: Ambari Server java process died with exitcode
255. Check /var/log/ambari-server/ambari-server.out for more information
由于是重新安装，所以在使用/etc/init.d/postgresql& initdb初始化数据库的时候会出现这个错误，所以需要
先用yum –y remove postgresql*命令把postgresql卸载
然后把/var/lib/pgsql/data目录下的文件全部删除
然后再配置postgresql数据库（执行1.6章节内容）
然后再次安装（3章节内容）
5.1.2在日志中有如下错误：ERROR [main] AmbariServer:820 - Failed to run
the Ambari Server
com.google.inject.ProvisionException: Guice
provision errors:
1) Error injecting method,
java.lang.NullPointerException
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:243)
org.apache.ambari.server.api.services.AmbariMetaInfo.class(AmbariMetaInfo.java:125)
while locating org.apache.ambari.server.api.services.AmbariMetaInfo
for field at
org.apache.ambari.server.controller.AmbariServer.ambariMetaInfo(AmbariServer.java:145)
org.apache.ambari.server.controller.AmbariServer.class(AmbariServer.java:145)
while locating org.apache.ambari.server.controller.AmbariServer
at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:813)
Caused by: java.lang.NullPointerException
org.apache.ambari.server.stack.StackModule.processRepositories(StackModule.java:665)
org.apache.ambari.server.stack.StackModule.resolve(StackModule.java:158)
at org.apache.ambari.server.stack.StackManager.fullyResolveStacks(StackManager.java:201)
org.apache.ambari.server.stack.StackManager.(StackManager.java:119)
org.apache.ambari.server.stack.StackManager$$FastClassByGuice$$33e4ffe0.newInstance()
&&&&&&& at
com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)
at com.sun.proxy.$Proxy26.create(Unknown Source)
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:247)
5.2安装HDFS和HBASE的时候出现/usr/hdp/current/hadoop-client/conf &doesn't exist
5.2.1/etc/Hadoop/conf文件链接存在
是由于/etc/hadoop/conf和/usr/hdp/current/hadoop-client/conf目录互相链接，造成死循环，所以要改变一个的链接
cd /etc/hadoop
rm -rf conf
ln -s /etc/hadoop/conf.backup
/etc/hadoop/conf
HBASE也会遇到同样的问题，解决方式同上
cd /etc/hbase
rm -rf conf
ln -s /etc/hbase/conf.backup /etc/hbase/conf
ZooKeeper也会遇到同样的问题，解决方式同上
cd /etc/zookeeper
rm -rf conf
ln -s /etc/zookeeper/conf.backup
/etc/zookeeper/conf
5.2.2/etc/Hadoop/conf文件链接不存在
查看正确的配置，发现缺少两个目录文件config.backup和2.4.0.0-169，把文件夹拷贝到/etc/hadoop目录下
重新创建/etc/hadoop目录下的conf链接：
/etc/hadoop
rm -rf conf
/usr/hdp/current/hadoop-client/conf conf
5.3在认证机器(Confirm
Hosts)的时候出现错误Ambari agent machine hostname (localhost) does not match expected
ambari server hostname
Ambari配置时在Confirm Hosts的步骤时，中间遇到一个很奇怪的问题：总是报错误：
Ambari agent machine hostname
(localhost.localdomain) does not match expected ambari server hostname (xxx).
后来修改的/etc/hosts文件中
127.0.0.1&& localhost dsj-kj1
::1&&&&&&&& localhost dsj-kj1
10.13.39.32&&&& dsj-kj1
10.13.39.33&&&& dsj-kj2
10.13.39.34&&&& dsj-kj3
10.13.39.35 &&&&dsj-kj4
10.13.39.36&&&& dsj-kj5
127.0.0.1& localhost
localhost.localdomain localhost4 localhost4.localdomain4
::1&&&&&&&& &localhost localhost.localdomain localhost6
localhost6.localdomain6
10.13.39.32&&&& dsj-kj1
10.13.39.33&&&& dsj-kj2
10.13.39.34&&&& dsj-kj3
10.13.39.35&&&& dsj-kj4
10.13.39.36&&&& dsj-kj5
感觉应该是走的ipv6协议，很奇怪，不过修改后就可以了。
5.4ambary-server重装
删除使用脚本删除
注意删除后要安装两个系统组件
yum -y install ruby*
yum -y install redhat-lsb*
yum -y install snappy*
5.5Ambari连接mysql设置
在主节点把mysql数据库连接包拷贝在/var/lib/ambary-server/resources目录下并改名为mysql-jdbc-driver.jar
/usr/share/java/mysql-connector-java-5.1.17.jar
/var/lib/ambari-server/resources/mysql-jdbc-driver.jar
再在图形界面下启动hive
5.6在注册机器(Confirm Hosts)的时候出现错误Failed
to start ping port listener of: [Errno 98] Address already in use
某个端口或者进程一直陪占用
解决方法：
发现df命令一直执行没有完成，
[root@testserver1 ~]# netstat -lanp|grep 8670
tcp&&&&&&& 0&&&&& 0
0.0.0.0:8670&&&&&&&&&&&&&&&
0.0.0.0:*&&&&&&&&&&&&&&&&&&
LISTEN&&&&& 2587/df
[root@testserver1 ~]# kill -9 2587
kill后，再重启ambari-agent问题解决
[root@testserver1 ~]# service ambari-agent restart
Verifying Python version compatibility...
Using python& /usr/bin/python2.6
ambari-agent is not running. No PID found at
/var/run/ambari-agent/ambari-agent.pid
Verifying Python version compatibility...
Using python& /usr/bin/python2.6
Checking for previously running Ambari Agent...
Starting ambari-agent
Verifying ambari-agent process status...
Ambari Agent successfully started
Agent PID at: /var/run/ambari-agent/ambari-agent.pid
Agent out at: /var/log/ambari-agent/ambari-agent.out
Agent log at: /var/log/ambari-agent/ambari-agent.log
5.7在注册机器(Confirm Hosts)的时候出现错误The following hosts have Transparent HugePages (THP) enabled。THP should
be disabled to avoid potential Hadoop performance issues
解决方法：
在Linux下执行：
echo never
&/sys/kernel/mm/redhat_transparent_hugepage/defrag
echo never &/sys/kernel/mm/redhat_transparent_hugepage/enabled
echo never &/sys/kernel/mm/transparent_hugepage/enabled
echo never &/sys/kernel/mm/transparent_hugepage/defrag
5.8启动hive的时候出现错误unicodedecodeerror ambari in position 117
查看/etc/sysconfig/i18n文件，发现内容如下：
LANG=”zh_CN.UTF8”
原来系统字符集设置成了中文，改成如下内容，问题解决：
LANG="en_US.UTF-8"
5.9安装Metrics的时候报如下错误，安装包找不到
1.failure: Updates-ambari-2.2.1.0/ambari/ambari-metrics-monitor-2.2.1.0-161.x86_64.rpm
from HDP-UTILS-1.1.0.20: [Errno 256] No more mirrors to try.
在ftp源服务器上执行命令：
cd /var/www/html/ambari/HDP-UTILS-1.1.0.20/repos/centos6
mkdir Updates-ambari-2.2.1.0
cp -r /var/www/html/ambari/Updates-ambari-2.2.1.0/ambari
/var/www/html/ambari/HDP-UTILS-1.1.0.20/repos/centos6/Updates-ambari-2.2.1.0
然后重新生成repodata
cd /var/www/html/ambari
rm -rf repodata
createrepo ./
2.failure: HDP-UTILS-1.1.0.20/repos/centos6/Updates-ambari-2.2.1.0/ambari/ambari-metrics-monitor-2.2.1.0-161.x86_64.rpm
from HDP-UTILS-1.1.0.20: [Errno 256] No more mirrors to try.
在/etc/yum.repos.d目录下删除mnt.repo，并使用yum clean all命令来清空yum的缓存
cd /ec/yum.repos.d
rm -rf mnt.repo
yum clean all
5.11jps 报process information unavailable解决办法
4791 -- process information unavailable
解决办法：
进入tmp目录，
删除该目录下
名称为hsperfdata_{ username}的文件夹
然后jps,清净了。
ls -l | grep hsperf | xargs rm
ls -l | grep hsperf
5.12namenode启动报错在日志文件中ERROR
namenode.NameNode (NameNode.java:main(1712)) - Failed to start namenode
日志中还有java.net.BindException: Port in use: gmaster:50070
Caused by: java.net.BindException: Address
already in use
判断原因是50070上一次没有释放，端口占用
netstat下time_wait状态的tcp连接：&
1.这是一种处于连接完全关闭状态前的状态；&
2.通常要等上4分钟（windows server）的时间才能完全关闭；&
3.这种状态下的tcp连接占用句柄与端口等资源，服务器也要为维护这些连接状态消耗资源；&
4.解决这种time_wait的tcp连接只有让服务器能够快速回收和重用那些TIME_WAIT的资源：修改注册表[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters]添加dword值TcpTimedWaitDelay=30（30也为微软建议值；默认为2分钟）和MaxUserPort：65534(可选值5000 - 65534)；&
5.具体tcpip连接参数配置还可参照这里：http://technet.microsoft.com/zh-tw/library/ccv=ws.10%29.aspx&
6.linux下：&
vi /etc/sysctl.conf&
新增如下内容：&
net.ipv4.tcp_tw_reuse
net.ipv4.tcp_tw_recycle = 1&
net.ipv4.tcp_syncookies=1&
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_keepalive_time=1800
net.ipv4.tcp_max_syn_backlog=8192
使内核参数生效：&
[root@web02 ~]# sysctl -p&
net.ipv4.tcp_syncookies=1 打开TIME-WAIT套接字重用功能，对于存在大量连接的Web服务器非常有效。&
net.ipv4.tcp_tw_recyle=1&
net.ipv4.tcp_tw_reuse=1 减少处于FIN-WAIT-2连接状态的时间，使系统可以处理更多的连接。&
net.ipv4.tcp_fin_timeout=30 减少TCP KeepAlive连接侦测的时间，使系统可以处理更多的连接。&
net.ipv4.tcp_keepalive_time=1800 增加TCP SYN队列长度，使系统可以处理更多的并发连接。&
net.ipv4.tcp_max_syn_backlog=8192
5.13在启动的时候报错误resource_management.core.exceptions.Fail:
Execution of 'ambari-sudo.sh& -H -E /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh
在日志中有如下内容：
13:55:28,090 INFO&
security.ShellBasedIdMapping
(ShellBasedIdMapping.java:updateStaticMapping(322)) - Not doing static UID/GID
mapping because '/etc/nfs.map' does not exist.
13:55:28,096 INFO&
nfs3.WriteManager (WriteManager.java:(92)) - Stream timeout
is 600000ms.
13:55:28,096 INFO&
nfs3.WriteManager (WriteManager.java:(100)) - Maximum open
streams is 256
13:55:28,096 INFO&
nfs3.OpenFileCtxCache (OpenFileCtxCache.java:(54)) - Maximum
open streams is 256
13:55:28,259 INFO&
nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:(205)) - Configured
HDFS superuser is
13:55:28,261 INFO&
nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:clearDirectory(231)) - Delete
current dump directory /tmp/.hdfs-nfs
13:55:28,269 WARN& fs.FileUtil
(FileUtil.java:deleteImpl(187)) - Failed to delete file or dir
[/tmp/.hdfs-nfs]: it still exists.
说明hdfs这个用户对/tmp没有权限
赋予权限给hdfs用户：
chown &hdfs:hadoop /tmp
再启动问题解决
5.14在安装ranger组件的时候出现错误连接不上mysql数据库rangeradmin用户和不能赋权的问题
在数据库中先删除所有rangeradmin用户，注意使用drop user命令：
drop user 'rangeradmin'@'%';
drop user 'rangeradmin'@'localhost';
drop user 'rangeradmin'@'gmaster';
drop user 'rangeradmin'@'gslave1';
drop user 'rangeradmin'@'gslave2';
FLUSH PRIVILEGES;
再创建用户（注意gmaster是ranger安装的服务器机器名）
CREATE USER 'rangeradmin'@'%' IDENTIFIED BY
'rangeradmin';
GRANT ALL PRIVILEGES ON *.* TO
'rangeradmin'@'%'& with grant option;
CREATE USER 'rangeradmin'@'localhost'
IDENTIFIED BY 'rangeradmin';
GRANT ALL PRIVILEGES ON *.* TO
'rangeradmin'@'localhost'& with grant option;
CREATE USER 'rangeradmin'@'gmaster'
IDENTIFIED BY 'rangeradmin';
GRANT ALL PRIVILEGES ON *.* TO
'rangeradmin'@'gmaster'& with grant option;
FLUSH PRIVILEGES;
再查看权限：
SELECT DISTINCT CONCAT('User:
''',user,'''@''',host,''';') AS query FROM mysql.user
select * from mysql.user where
user='rangeradmin' \G;
5.15在ambari启动的时候出现错误：AmbariServer:820
- Failed to run the Ambari Server
这个问题困扰了我很久，最后通过查看源码找到了问题所在：
在/var/log/ambari-server/ambary-server.log文件中报有错误：
13 Apr :01,723& INFO [main] StackDirectory:458 - Stack
'/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS' doesn't contain an
upgrade directory
13 Apr :01,723& INFO [main] StackDirectory:468 - Stack
'/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS' doesn't contain
config upgrade pack file
13 Apr :01,744& INFO [main] StackDirectory:484 - Role command
order info was loaded from file:
/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS/role_command_order.json
13 Apr :01,840& INFO [main] StackDirectory:484 - Role command
order info was loaded from file:
/var/lib/ambari-server/resources/stacks/HDP/2.4/role_command_order.json
13 Apr :01,927 ERROR [main]
AmbariServer:820 - Failed to run the Ambari Server
com.google.inject.ProvisionException: Guice
provision errors:
1) Error injecting method,
java.lang.NullPointerException
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:243)
org.apache.ambari.server.api.services.AmbariMetaInfo.class(AmbariMetaInfo.java:125)
while locating org.apache.ambari.server.api.services.AmbariMetaInfo
for field at
org.apache.ambari.server.controller.AmbariServer.ambariMetaInfo(AmbariServer.java:145)
org.apache.ambari.server.controller.AmbariServer.class(AmbariServer.java:145)
while locating org.apache.ambari.server.controller.AmbariServer
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
&&&&&&&& at
org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:813)
Caused by: java.lang.NullPointerException
&&&&&&&& at
org.apache.ambari.server.stack.StackModule.processRepositories(StackModule.java:665)
&&&&&&&& at
org.apache.ambari.server.stack.StackModule.resolve(StackModule.java:158)
&&&&&&&& at
org.apache.ambari.server.stack.StackManager.fullyResolveStacks(StackManager.java:201)
&&&&&&&& at
org.apache.ambari.server.stack.StackManager.(StackManager.java:119)
&&&&&&&& at
org.apache.ambari.server.stack.StackManager$$FastClassByGuice$$33e4ffe0.newInstance()
&&&&&&&& at
com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
&&&&&&&& at
com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
&&&&&&&& at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
&&&&&&&& at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
&&&&&&&& at
com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)
&&&&&&&& at
com.sun.proxy.$Proxy26.create(Unknown Source)
&&&&&&&& at
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:247)
&&&&&&&& at
org.apache.ambari.server.api.services.AmbariMetaInfo$$FastClassByGuice$$202844bc.invoke()
&&&&&&&& at
com.google.inject.internal.cglib.reflect.$FastMethod.invoke(FastMethod.java:53)
&&&&&&&& at
com.google.inject.internal.SingleMethodInjector$1.invoke(SingleMethodInjector.java:56)
&&&&&&&& at
com.google.inject.internal.SingleMethodInjector.inject(SingleMethodInjector.java:90)
&&&&&&&& at
com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
&&&&&&&& at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
&&&&&&&& at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
&&&&&&&& at
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
&&&&&&&& at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
&&&&&&&& at
com.google.inject.Scopes$1$1.get(Scopes.java:65)
&&&&&&&& at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
&&&&&&&& at
com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
&&&&&&&& at
com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
&&&&&&&& at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
&&&&&&&& at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
&&&&&&&& at
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
&&&&&&&& at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
&&&&&&&& at
com.google.inject.Scopes$1$1.get(Scopes.java:65)
&&&&&&&& at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
&&&&&&&& at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
&&&&&&&& at
com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
&&&&&&&& ...
解决方法：
后来发现在文件/var/lib/ambari-server/resources/stacks/HDP/2.4/repos/repoinfo.xml中的内容os这行原来的如下：
5.16在启动hive的时候出现错误Error:
Duplicate key name 'PCS_STATS_IDX' (state=42000,code=1061)
在HIVE安装的机器cd /var/lib/ambari-agent/data目录下有日志，日志中报如下错误：
Traceback (most recent call last):
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 245, in
HiveMetastore().execute()
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 219, in execute
method(env)
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 58, in start
self.configure(env)
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 72, in configure
hive(name = 'metastore')
"/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py",
line 89, in thunk
return fn(*args, **kwargs)
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py",
line 292, in hive
user = params.hive_user
"/usr/lib/python2.6/site-packages/resource_management/core/base.py",
line 154, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 158, in run
self.run_action(resource, action)
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 121, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
line 238, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 70, in inner
result = function(command, **kwargs)
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 92, in checked_call
tries=tries, try_sleep=try_sleep)
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail:
Execution of 'export
HIVE_CONF_DIR=/usr/hdp/current/hive-metastore/conf/conf.
/usr/hdp/current/hive-metastore/bin/schematool -initSchema -dbType mysql
-userName hive -passWord [PROTECTED]' returned 1. WARNING: Use "yarn
jar" to launch YARN applications.
connection URL:&&&&&&&
jdbc:mysql://a2slave1/hive?createDatabaseIfNotExist=true
Connection Driver :&&&
com.mysql.jdbc.Driver
connection User:&&&&&& hive
metastore schema initialization to 1.2.1000
Initialization
script hive-schema-1.2.1000.mysql.sql
Duplicate key name 'PCS_STATS_IDX' (state=42000,code=1061)
org.apache.hadoop.hive.metastore.HiveMetaException:
Schema initialization FAILED! Metastore state would be inconsistent !!
schemaTool failed ***
解决方法：
HIVE安装的机器拷贝脚本hive-schema-1.2.1000.mysql.sql到：
[root@a2master /]# scp /usr/hdp/2.4.0.0-169/hive/scripts/metastore/upgrade/mysql/hive-schema-1.2.1000.mysql.sql root@a2slave1:/usr/local/mysql
hive-schema-1.2.1000.mysql.sql&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
100%&& 34KB& 34.4KB/s&&
在HIVE安装机器使用hive用户登陆，进入hive数据库，执行这个脚本：
[root@a2slave1 conf.server]# mysql -uhive
Enter password:
Welcome to the MySQL monitor.& C or \g.
Your MySQL connection id is 505
Server version: 5.6.26-log MySQL Community
Server (GPL)
Copyright (c) , Oracle and/or its
affiliates. All rights reserved.
Oracle is a registered trademark of Oracle
Corporation and/or its
affiliates. Other names may be trademarks
of their respective
Type '' or '\h' for help. Type '\c' to
clear the current input statement.
Reading table information for completion of
table and column names
You can turn off this feature to get a
quicker startup with -A
Database changed
mysql& source
/usr/local/mysql/hive-schema-1.2.1000.mysql.
5.17在启动hive的时候出现错误Could
not create "increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.
在/var/log/hive中的日志文件hiveserver.log中记录有：
10:45:20,446 INFO& [main]: server.HiveServer2
(HiveServer2.java:startHiveServer2(405)) - Starting HiveServer2
10:45:20,573 INFO& [main]: metastore.ObjectStore
(ObjectStore.java:initialize(294)) - ObjectStore, initialize called
10:45:20,585 INFO& [main]: metastore.MetaStoreDirectSql
(MetaStoreDirectSql.java:(140)) - Using direct SQL, underlying DB
10:45:20,585 INFO& [main]: metastore.ObjectStore (ObjectStore.java:setConf(277))
- Initialized ObjectStore
10:45:20,590 WARN& [main]: metastore.ObjectStore
(ObjectStore.java:getDatabase(577)) - Failed to get database default, returning
NoSuchObjectException
10:45:20,591 ERROR [main]: bonecp.ConnectionHandle
(ConnectionHandle.java:markPossiblyBroken(388)) - Database access problem.
Killing off this connection and all remaining connections in the connection
pool. SQL State = HY000
10:45:20,600 WARN& [main]: metastore.HiveMetaStore
(HiveMetaStore.java:createDefaultDB(623)) - Retrying creating default database
after error: Could not create "increment"/"table"
value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow
javax.jdo.JDOUserException: Could not create
"increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.
&&&&&&&& at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:549)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
&&&&&&&& at
org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)
&&&&&&&& at
sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
&&&&&&&& at
com.sun.proxy.$Proxy6.createDatabase(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:621)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
&&&&&&&& at
sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)
&&&&&&&& at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.init(CLIService.java:112)
&&&&&&&& at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.util.RunJar.run(RunJar.java:221)
&&&&&&&& at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)
NestedThrowablesStackTrace:
Could not create
"increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.
org.datanucleus.exceptions.NucleusUserException:
Could not create "increment"/"table" value-generation
container `SEQUENCE_TABLE` since autoCreate flags do not allow it.
&&&&&&&& at
org.datanucleus.store.rdbms.valuegenerator.TableGenerator.createRepository(TableGenerator.java:261)
&&&&&&&& at
org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obtainGenerationBlock(AbstractRDBMSGenerator.java:162)
&&&&&&&& at
org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerationBlock(AbstractGenerator.java:197)
&&&&&&&& at
org.datanucleus.store.valuegenerator.AbstractGenerator.next(AbstractGenerator.java:105)
&&&&&&&& at
org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueForGenerator(RDBMSStoreManager.java:2005)
&&&&&&&& at
org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1386)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3827)
&&&&&&&& at
org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManager.java:2571)
&&&&&&&& at
org.datanucleus.state.JDOStateManager.initialiseForPersistentNew(JDOStateManager.java:513)
&&&&&&&& at
org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:232)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.newObjectProviderForPersistentNew(ExecutionContextImpl.java:1414)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2218)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913)
&&&&&&&& at
org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
&&&&&&&& at
org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)
&&&&&&&& at
sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
&&&&&&&& at
com.sun.proxy.$Proxy6.createDatabase(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:621)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
&&&&&&&& at
sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)
&&&&&&&& at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.init(CLIService.java:112)
&&&&&&&& at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.util.RunJar.run(RunJar.java:221)
&&&&&&&& at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)
10:45:20,607 WARN& [main]: metastore.ObjectStore
(ObjectStore.java:getDatabase(577)) - Failed to get database default, returning
NoSuchObjectException
10:45:20,609 ERROR [main]:
bonecp.ConnectionHandle (ConnectionHandle.java:markPossiblyBroken(388)) -
Database access problem. Killing off this connection and all remaining
connections in the connection pool. SQL State = HY000
10:45:20,617 INFO& [main]: server.HiveServer2
(HiveServer2.java:stop(371)) - Shutting down HiveServer2
10:45:20,618 WARN& [main]: server.HiveServer2
(HiveServer2.java:startHiveServer2(442)) - Error starting HiveServer2 on
attempt 29, will retry in 60 seconds
java.lang.RuntimeException: Error applying
authorization policy on hive configuration: java.lang.RuntimeException: Unable
to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
&&&&&&&& at
org.apache.hive.service.cli.CLIService.init(CLIService.java:114)
&&&&&&&& at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)
&&&&&&&& at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.util.RunJar.run(RunJar.java:221)
&&&&&&&& at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException:
java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
&&&&&&&& at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:494)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)
&&&&&&&& at
org.apache.hive.service.cli.CLIService.init(CLIService.java:112)
&&&&&&&& ...
Caused by: java.lang.RuntimeException:
Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
&&&&&&&& at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1533)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)
&&&&&&&& at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)
&&&&&&&& ...
Caused by:
java.lang.reflect.InvocationTargetException
&&&&&&&& at
sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)
&&&&&&&& ...
Caused by: javax.jdo.JDOUserException:
Could not create "increment"/"table" value-generation
container `SEQUENCE_TABLE` since autoCreate flags do not allow it.
NestedThrowables:
org.datanucleus.exceptions.NucleusUserException:
Could not create "increment"/"table" value-generation
container `SEQUENCE_TABLE` since autoCreate flags do not allow it.
&&&&&&&& at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:549)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
&&&&&&&& at
org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)
&&&&&&&& at
sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
&&&&&&&& at
com.sun.proxy.$Proxy6.createDatabase(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:625)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)
&&&&&&&& at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)
&&&&&&&& at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)
&&&&&&&& at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
&&&&&&&& ...
Caused by:
org.datanucleus.exceptions.NucleusUserException: Could not create
"increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.
&&&&&&&& at
org.datanucleus.store.rdbms.valuegenerator.TableGenerator.createRepository(TableGenerator.java:261)
&&&&&&&& at
org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obtainGenerationBlock(AbstractRDBMSGenerator.java:162)
&&&&&&&& at
org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerationBlock(AbstractGenerator.java:197)
&&&&&&&& at
org.datanucleus.store.valuegenerator.AbstractGenerator.next(AbstractGenerator.java:105)
&&&&&&&& at
org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueForGenerator(RDBMSStoreManager.java:2005)
&&&&&&&& at
org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1386)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3827)
&&&&&&&& at
org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManager.java:2571)
&&&&&&&& at
org.datanucleus.state.JDOStateManager.initialiseForPersistentNew(JDOStateManager.java:513)
&&&&&&&& at
org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:232)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.newObjectProviderForPersistentNew(ExecutionContextImpl.java:1414)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2218)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)
&&&&&&&& at
org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913)
&&&&&&&& at
org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
&&&&&&&& at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727)
&&&&&&&& ...
最后发现是mysql数据库的binlog_format参数设置不正确，
原来设置的是STATEMENT，修改为MIXED
修改方法，在/etc/my.cnf文件中加上binlog_format=MIXED
然后重启mysql数据库
5.18在使用hive命令进入hive的时候报错误Permission
denied: user=root, access=WRITE,
inode="/user/root":hdfs:hdfs:drwxr-xr-x& hive
解决方法：
1.&&&&&& 使用HDFS的命令行接口修改相应目录的权限，hadoop fs -chmod 777 /user,后面的/user是要上传文件的路径，不同的情况可能不一样，比如要上传的文件路径为hdfs://namenode/user/xxx.doc，则这样的修改可以，如果要上传的文件路径为hdfs://namenode/java/xxx.doc，则要修改的为hadoop
fs -chmod 777 /java或者hadoop fs -chmod 777 /，java的那个需要先在HDFS里面建立Java目录，后面的这个是为根目录调整权限。
hadoop fs -chmod 777 /user
2.&&&&&& 在/etc/profile文件中加上系统的环境变量或java JVM变量里面添加export HADOOP_USER_NAME=hdfs(ambari使用的hadoop用户是hdfs)，这个值具体等于多少看自己的情况，以后会运行HADOOP上的Linux的用户名。
export HADOOP_USER_NAME=hdfs
5.19Spark Thrift
Server组件启动后自动关闭
Spark Thrift Server组件在启动后自动关闭，查看/var/log/spark下的日志文件spark-hive-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-a2master.out，中有如下内容：
10:26:10 WARN DomainSocketFactory: The short-circuit local reads feature cannot
be used because libhadoop cannot be loaded.
10:26:10 INFO Client: Requesting a new application from cluster with 3
NodeManagers
10:26:10 INFO Client: Verifying our application has not requested more than the
maximum memory capability of the cluster (512 MB per container)
10:26:10 ERROR SparkContext: Error initializing SparkContext.
java.lang.IllegalArgumentException: Required executor memory ( MB) is
above the max threshold (512 MB) of this cluster! Please check the values of
'yarn.scheduler.maximum-allocation-mb' and/or
'yarn.nodemanager.resource.memory-mb'.
&&&&&&& at
org.apache.spark.deploy.yarn.Client.verifyClusterResources(Client.scala:283)
&&&&&&& at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:139)
&&&&&&& at
org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
&&&&&&& at
org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
&&& &&&&at
org.apache.spark.SparkContext.(SparkContext.scala:530)
&&&&&&& at
org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:56)
&&&&&&& at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:76)
&&&&&&& at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
&&&&&&& at
org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
16/04/18 10:26:11 INFO ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages/stage/kill,null}
分析：原因是spark的内存设置的太小
解决方法：在前台界面修改内存为1536M，并把配置在其它机器上更新，重启spark服务
5.20hbase在启动的时候出现错误Permission
denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,0860883":hdfs:hdfs:drwxr-xr-x
在日志中有如下内容：
15:42:11,640 INFO& [regionserver/gslave2/192.168.1.253:16020]
hfile.CacheConfig: Allocating LruBlockCache size=401.60 MB, blockSize=64 KB
15:42:11,648 INFO& [regionserver/gslave2/192.168.1.253:16020]
hfile.CacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=433080,
freeSize=, maxSize=, heapSize=433080, minSize=,
minFactor=0.95, multiSize=, multiFactor=0.5, singleSize=,
singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false,
cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false,
cacheDataCompressed=false, prefetchOnOpen=false
15:42:11,704 INFO& [regionserver/gslave2/192.168.1.253:16020]
wal.WALFactory: Instantiating WALProvider of type class
org.apache.hadoop.hbase.wal.DefaultWALProvider
15:42:11,729 INFO& [regionserver/gslave2/192.168.1.253:16020]
regionserver.HRegionServer: STOPPED: Failed initialization
15:42:11,729 ERROR
[regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: Failed
org.apache.hadoop.security.AccessControlException:
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
&&&&&&&& at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
&&&&&&&& at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
&&&&&&&& at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2589)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2558)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:820)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:816)
&&&&&&&& at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:816)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:809)
&&&&&&&& at
org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:488)
&&&&&&&& at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
&&&&&&&& at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
&&&&&&&& at
org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:179)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1624)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1362)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:899)
&&&&&&&& at
java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
&&&&&&&& at
org.apache.hadoop.ipc.Client.call(Client.java:1411)
&&&&&&&& at
org.apache.hadoop.ipc.Client.call(Client.java:1364)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
&&&&&&&& at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
&&&&&&&& at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
&&&&&&&& at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:508)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)
&&&&&&&& at
com.sun.proxy.$Proxy16.mkdirs(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2587)
&&&&&&&& ...
15:42:11,732 FATAL
[regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: ABORTING
region server gslave2,8130424: Unhandled: Permission denied:
user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
org.apache.hadoop.security.AccessControlException:
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
&&&&&&&& at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
&&&&&&&& at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&&&&&&&& at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
&&&&&&&& at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
&&&&&&&& at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2589)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2558)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:820)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:816)
&&&&&&&& at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:816)
&&&&&&&& at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:809)
&&&&&&&& at
org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:488)
&&&&&&&& at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
&&&&&&&& at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
&&&&&&&& at
org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:179)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1624)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1362)
&&&&&&&& at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:899)
&&&&&&&& at
java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,8130424":hdfs:hdfs:drwxr-xr-x
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)
&&&&&&&& at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
&&&&&&&& at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&&&&&&&& at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
&&&&&&&& at
java.security.AccessController.doPrivileged(Native Method)
&&&&&&&& at
javax.security.auth.Subject.doAs(Subject.java:415)
&&&&&&&& at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
&&&&&&&& at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
&&&&&&&& at
org.apache.hadoop.ipc.Client.call(Client.java:1411)
&&&&&&&& at
org.apache.hadoop.ipc.Client.call(Client.java:1364)
&&&&&&&& at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
&&&&&&&& at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
&&&&&&&& at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
&&&&&&&& at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:508)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&&&&&&&& at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
&&&&&&&& at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&&&&&&&& at
java.lang.reflect.Method.invoke(Method.java:606)
&&&&&&&& at
org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)
&&&&&&&& at
com.sun.proxy.$Proxy16.mkdirs(Unknown Source)
&&&&&&&& at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2587)
&&&&&&&& ...
15:42:11,732 FATAL
[regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer:
RegionServer abort: loaded coprocessors are: []
15:42:11,744 INFO& [regionserver/gslave2/192.168.1.253:16020]
regionserver.HRegionServer: Dump of metrics as JSON on abort: {
解决方法：
hdfs dfs -chown -R hbase:hbase /apps/hbase
阅读(17472) | 评论(0) | 转发(0) |
相关热门文章
给主人留下些什么吧！~~
请登录后评论。}
我要回帖
更多关于 js key value 数组 的文章
更多推荐
- ·mhsmhserver是什么软件？
- ·如何解决微信不能分身应用出现异常怎么办的问题。
- ·欲贷款买车，哥瑞，思域，本田和丰田15万左右车买哪款XR-V选哪个？
- ·qq音乐在哪里看QQ音乐活动中心心？
- ·不没戴头盔罚款20还是50处罚规定
- ·环球美联英语学费费一年下来要多少
- ·用xrf测试仪金属成分为什么不准确
- ·红米note3全网通root双网通的卡槽支不支持内存卡
- ·健忘村迅雷下载mp4下载迅雷
- ·为什么我天天酷跑微信和qq数据电脑版无法用qq登录
- ·现代的显卡自带声卡是不是都附带声卡功能
- ·精英版的新飞度换鸣笛喇叭叭是单插还是双插的
- ·网易严选产品分析曾经的产品补货快吗
- ·二次元软件必备的手机软件有哪些
- ·win10麦克风声音小的声音小怎么办win10
- ·王者荣耀陈赫直播回放六整场回放谁知道在哪里看
- ·烧水壶镇雷下桌子电脑的声音就完全没了这样的话 英语怎么修
- ·dependent data和independent datadata code是什么意思思
- ·求dos批处理 批量重命名重命名文件夹名
- ·分区后会硬盘格式化分区全部硬盘吗
- ·key获取json的key和valueemaxsize最大多少
- ·熊猫3631b海信彩电高频头只有一个高频头接口现要与机电盒相连接怎么连
- ·小臭臭致六岁哪个致命复活 西瓜播放器器有
- ·電腦QQ為台湾用什麼社交软件業面這麼小
- ·ps vr htc viver 支不支持fozur软件
- ·ubuntu nginx php上怎么用nginx支持php
- ·华为手机微信红包表情动态图像打开怎么一条黑线像是进水似的
- ·苹果手机国外直播软件有什么可以看国外视频的软件
- ·网吧的这鼠标谁知道牧马人鼠标是什么牌子子的
- ·网络掉线是谁的问题，联通光猫无线信号不好好，路由器不好，光猫不好
- ·如何在google map api 搜索Flash API For Map上添加ScreenOverlay
- ·mca-我的世界1.10.2材质包-5.2.1-universal怎么结婚
- ·有没有好用的录播软件推荐一下的.能录腾讯课堂录播内容
- ·怎么恢复出厂设置密码成“无密码”
- ·我键盘pause break上没有PAUSE键，怎么开始
