kylin的安装

在[官网](https://kylin.apache.org/download/)下载或在国内其他源内下载 我下载的版本[apache-kylin-3.1.3-bin-hbase1x](https://dlcdn.apache.org/kylin/apache-kylin-3.1.3/apache-kylin-3.1.3-bin-hbase1x.tar.gz) # 安装 上传到服务器,工具不限 解压: `tar -xvf apache-kylin-3.1.3-bin-hbase1x.tar.gz` 改名: `mv apache-kylin-3.1.3-bin-hbase1x.tar.gz kylin` 移动到安装目录: `mv kylin /opt` 配置: 不搭集群不用配置也能用 # 以下=====区间的内容可忽略,待整理,报错看最后 # ================================== # 配置 修改HIVE的配置文件 修改hive根目录下的conf下的`hive-site.xml` 在<configuration></configuration>内添加: ```xml <!--指定hive.metastore.uris的port.为了启动metastore服务的时候不用指定端口--> <! --hive ==service metastore -p 9083 & L hive ==service metastore--> <property> <name>hive.metastore.uris</name> <value>thrift://node01:9083</value> </property> ``` 修改HBase配置文件 修改HBASE的根目录下的conf下的`hbase-site.xml` 在<configuration></configuration>内添加或修改: ```xml <!-- zookeeper集群的URL配置多个host中间用逗号--> <property> <name>hbase.zookeeper.quorum</name> <value>node01,node02,node03</value> </property> ``` HBase修改$HBASE_HOME$​/bin/hbase 的 classpat 在hbase/bin下的hbase内,将: ``` # CLASSPATH initially contains $HBASE_CONF_DIR CLASSPATH="${HBASE_CONF_DIR}" CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar ``` 修改为: ``` # CLASSPATH initially contains $HBASE_CONF_DIR CLASSPATH="${HBASE_CONF_DIR}" CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar:$HBASE_HOME/lib/* ``` 配置环境变量: 在`vi /etc/profile`内添加: ``` # kylin 安装地址 # 路径换成自己的安装路径 export KYLIN_HOME=/opt/kylin export PATH=$KYLIN_HOME/bin:$PATH export HCAT_HOME=$HIVE_HOME/hcatalog export PATH=$HCAT_HOME/bin:$PATH export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop export HIVE_CONF_HOME=$HIVE_HOME/conf ``` 更新配置 `source /etc/profile` 注:`Kylin 会自动从环境中读取 Hadoop 配置( core-site.xml ), Hive 配置( hivesite.xml )和 HBase 配置( hbase-site.xml )` # 修改配置文件 修改 `kylin.properties` 配置文件。此步骤在单节点时这一步可以忽略掉。 路径:kylin/conf/kylin.properties ```language #配置节点类型(kylin节点模式分为all、query(查询模式)、job(任务构建模式)){约47行} kylin.server.mode=all #kylin集群节点配置{约50行} kylin.server.cluster-servers=node01:7070,node02:7070,node03:7070 ``` # 拷贝kylin到其他节点 ```language scp -r /opt/kylin hadoop-02:/opt scp -r /opt/kylin hadoop-03:/opt ``` # 修改其他节点的配置 路径:kylin/conf/kylin.properties ```language #配置节点类型 query(查询模式){约47行} kylin.server.mode=query ``` # 删除所有数据 `kylin.sh org.apache.kylin.tool.StorageCleanupJob --force true --delete true` # ================================= # 报错 启动时报:hbase-common lib not found 解决办法:将HBase根目录下的lib里面的hbase-common-*.*.*.jar添加到环境变量 export HBASE_CLASSPATH=$HBASE_HOME/lib/hbase-common-2.4.13.jar 正常启动后,web页面HTTP Status 404 – 未找到,查看kylin/logs/kylin.log日志,发现如下错误: ```shell ERROR [localhost-startStop-1] context.ContextLoader:350 : Context initialization failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter]: Constructor threw exception; nested exception is java.lang.ClassCastException: com.fasterxml.jackson.datatype.jsr310.JavaTimeModule cannot be cast to com.fasterxml.jackson.databind.Module at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1160) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1104) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443) at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325) at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705) at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:970) at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1841) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ``` 进入kylin/bin 修改:find-spark-dependency.sh 添加`! -name '*jackson*' !` ![image.png](https://cos.easydoc.net/52087651/files/l5x61gll.png) 修改:find-hive-dependency.sh 添加`! -name '*jackson*' !` ![image.png](https://cos.easydoc.net/52087651/files/l5x6bre7.png) 删除之前的缓存 还是在kylin/bin目录下:`rm cached-*` ![image.png](https://cos.easydoc.net/52087651/files/l5x6fiwc.png) 重启kylin即可 遇到: ```log org.apache.kylin.engine.mr.exception.MapReduceException: no counters for job job_1663636064241_0146Job Diagnostics: Failure task Diagnostics: Error: Java heap space ``` 在linux执行如下代码,似乎没有作用 ```bash set io.sort.mb=10; ``` 可以再尝试 `kylin.properties` 内添加: ```shell # kylin.properties kylin.engine.mr.config-override.mapreduce.map.java.opts=-Xmx8g kylin.engine.mr.config-override.mapreduce.map.memory.mb=10240 ``` 如果报: ```log killed by admin ```