liwei131313

第 1725 位会员
注册于 2018-02-26 14:52:07
活跃于 2018-06-04 10:40:15
最近话题
最新评论
  • 信息被删除或无权限查看
  • 信息被删除或无权限查看
  • 信息被删除或无权限查看
  • 海牛部落 hive 系列教程（十九）：hive 介绍与安装 at 2018-04-28 14:11:05

    学习

  • 海牛部落 hive 系列教程（二十一）：hive 的 orc 文件、bucket 使用、表操作、关联查询 at 2018-04-28 11:27:09

    谢谢

  • hive 和 hbase 整合，往分区表插入数据报错？ at 2018-03-22 16:25:28

    @青牛
    加上where不行，会报错。
    hive> insert overwrite table hbase_table_1 partition (day='2012-01-02') select * from pokes where foo=4;
    Query ID = root_20180322162424_577330d0-5b03-4d17-92be-552eb126557d
    Total jobs = 3
    Launching Job 1 out of 3
    Number of reduce tasks is set to 0 since there's no reduce operator
    java.io.IOException: java.lang.IllegalArgumentException: Must specify table name
    at org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(FileSinkOperator.java:1080)
    at org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl.checkOutputSpecs(HiveOutputFormatImpl.java:67)
    at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:272)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:143)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:578)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:573)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:573)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:564)
    at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:428)
    at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:142)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1978)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1691)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1423)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1207)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1197)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:226)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:389)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:699)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:634)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
    Caused by: java.lang.IllegalArgumentException: Must specify table name
    at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:195)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
    at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:277)
    at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:267)
    at org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(FileSinkOperator.java:1078)
    ... 37 more
    Job Submission failed with exception 'java.io.IOException(java.lang.IllegalArgumentException: Must specify table name)'
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

  • hive 和 hbase 整合，往分区表插入数据报错？ at 2018-03-22 15:37:31

    @青牛
    添加limit后，就可以，数据可以插入到分区表，这个是什么原因导致的？如果需要将源表中的数据全量插入到指定分区，要如何处理呢？

    file

  • 在 impala 的 shell 命令窗口，执行删除表的命令，执行报错，表无法删除？ at 2018-03-21 10:21:42

    @青牛
    查看详细的debug日志的话，还是看不出问题在哪里。
    第二天再次执行删除语句就正常了。
    部分debug日志信息如下：
    18/03/20 17:08:41 [main]: DEBUG zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false
    18/03/20 17:08:41 [main-EventThread]: INFO state.ConnectionStateManager: State change: CONNECTED
    18/03/20 17:09:02 [main]: ERROR exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:One or more instances could not be deleted)
    Caused by: MetaException(message:One or more instances could not be deleted)
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:One or more instances could not be deleted)
    18/03/20 17:09:02 [main]: ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:One or more instances could not be deleted)
    18/03/20 17:09:02 [main]: DEBUG metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith(HiveConf, )=0, getTable(String, String, )=9}

  • 在 impala 的 shell 命令窗口，执行删除表的命令，执行报错，表无法删除？ at 2018-03-20 16:16:49

    @青牛 把hdfs目录删除，再执行删除表语句还是报错。

  • 在 impala 的 shell 命令窗口，执行删除表的命令，执行报错，表无法删除？ at 2018-03-20 16:00:56

    @青牛 没有分区，就是一张普通的表

  • 在 impala 的 shell 命令窗口中修改表的字段类型，查询数据报错？ at 2018-03-13 13:42:35

    使用 Parquet 文件格式

  • 信息被删除或无权限查看
  • 信息被删除或无权限查看
  • 信息被删除或无权限查看
  • 信息被删除或无权限查看