APACHE-ATLAS-2.1.0 - 基础运维

news/2024/12/22 13:29:59/

(一)SOLR相关

1. 如何创建/删除集合?
# 1. 删除
solr/bin/solr delete -c vertex_index
solr/bin/solr delete -c edge_index
solr/bin/solr delete -c fulltext_index# 2. 创建
solr/bin/solr create -c vertex_index -force -d conf/solr/
solr/bin/solr create -c edge_index -force -d conf/solr/
solr/bin/solr create -c fulltext_index -force -d conf/solr/# 删除
[root@master apache-atlas-2.1.0]# solr/bin/solr delete -c vertex_index
INFO  - 2023-06-15 00:19:40.690; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
{"responseHeader":{"status":0,"QTime":835}}
Deleted collection 'vertex_index' using command:
http://192.168.118.174:8983/solr/admin/collections?action=DELETE&name=vertex_index
[root@master apache-atlas-2.1.0]# solr/bin/solr delete -c edge_index
INFO  - 2023-06-15 00:19:44.906; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
{"responseHeader":{"status":0,"QTime":634}}
Deleted collection 'edge_index' using command:
http://192.168.118.174:8983/solr/admin/collections?action=DELETE&name=edge_index
[root@master apache-atlas-2.1.0]# solr/bin/solr delete -c fulltext_index
INFO  - 2023-06-15 00:19:52.255; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
{"responseHeader":{"status":0,"QTime":707}}
Deleted collection 'fulltext_index' using command:
http://192.168.118.174:8983/solr/admin/collections?action=DELETE&name=fulltext_index
[root@master apache-atlas-2.1.0]# 
# 创建
[root@master apache-atlas-2.1.0]# solr/bin/solr create -c vertex_index -force -d conf/solr/
INFO  - 2023-06-15 00:20:04.079; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
Created collection 'vertex_index' with 1 shard(s), 1 replica(s) with config-set 'vertex_index'
[root@master apache-atlas-2.1.0]# solr/bin/solr create -c edge_index -force -d conf/solr/
INFO  - 2023-06-15 00:20:13.817; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
Created collection 'edge_index' with 1 shard(s), 1 replica(s) with config-set 'edge_index'
[root@master apache-atlas-2.1.0]# solr/bin/solr create -c fulltext_index -force -d conf/solr/
INFO  - 2023-06-15 00:20:21.018; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
Created collection 'fulltext_index' with 1 shard(s), 1 replica(s) with config-set 'fulltext_index'
[root@master apache-atlas-2.1.0]# 
2. SOLR的日志在哪里?
/opt/rills/software/apache-atlas-2.1.0/solr/server/logs/solr.log

(二)HBASE相关

1. 内置的HBASE无法启动

错误信息如下:
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures

2023-06-15 09:06:28,071 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to establish new session at /0:0:0:0:0:0:0:1:55222
2023-06-15 09:06:28,076 INFO  [SyncThread:0] server.ZooKeeperServer: Established session 0x188bc96d4a90002 with negotiated timeout 10000 for client /0:0:0:0:0:0:0:1:55222
2023-06-15 09:06:28,076 INFO  [ReadOnlyZKClient-localhost:2181@0x79f2feba-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x188bc96d4a90002, negotiated timeout = 10000
2023-06-15 09:06:28,207 INFO  [Thread-22] procedure2.ProcedureExecutor: Starting 16 core workers (bigger of cpus/4 or 16) with max (burst) worker count=160
2023-06-15 09:06:28,214 WARN  [Thread-22] wal.WALProcedureStore: Remove uninitialized log: DeprecatedRawLocalFileStatus{path=file:/opt/rills/software/apache-atlas-2.1.0/data/hbase-root/MasterProcWALs/pv2-00000000000000000002.log; isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1686790826000; access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false}
2023-06-15 09:06:28,215 INFO  [Thread-22] wal.ProcedureWALFile: Archiving file:/opt/rills/software/apache-atlas-2.1.0/data/hbase-root/MasterProcWALs/pv2-00000000000000000002.log to file:/opt/rills/software/apache-atlas-2.1.0/data/hbase-root/oldWALs/pv2-00000000000000000002.log
2023-06-15 09:06:28,275 ERROR [Thread-22] master.HMaster: Failed to become active master
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1044)at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:383)at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:649)at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1282)at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:842)at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2086)at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:553)at java.lang.Thread.run(Thread.java:748)
2023-06-15 09:06:28,276 ERROR [Thread-22] master.HMaster: ***** ABORTING master master,61500,1686791181740: Unhandled exception. Starting shutdown. *****
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1044)at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:383)at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:649)at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1282)at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:842)at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2086)at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:553)at java.lang.Thread.run(Thread.java:748)
2023-06-15 09:06:28,277 INFO  [Thread-22] regionserver.HRegionServer: ***** STOPPING region server 'master,61500,1686791181740' *****
2023-06-15 09:06:28,277 INFO  [Thread-22] regionserver.HRegionServer: STOPPED: Stopped by Thread-22
2023-06-15 09:06:28,841 INFO  [master/master:61500.splitLogManager..Chore.1] hbase.ScheduledChore: Chore: SplitLogManager Timeout Monitor was stopped
2023-06-15 09:06:30,484 INFO  [M:0;master:61500] ipc.NettyRpcServer: Stopping server on /192.168.118.174:61500
2023-06-15 09:06:30,497 INFO  [M:0;master:61500] regionserver.HRegionServer: Stopping infoServer
2023-06-15 09:06:30,538 INFO  [M:0;master:61500] handler.ContextHandler: Stopped o.e.j.w.WebAppContext@67440de6{/,null,UNAVAILABLE}{file:/opt/rills/software/apache-atlas-2.1.0/hbase/hbase-webapps/master}
2023-06-15 09:06:30,545 INFO  [M:0;master:61500] server.AbstractConnector: Stopped ServerConnector@3910fe11{HTTP/1.1,[http/1.1]}{0.0.0.0:61510}
2023-06-15 09:06:30,547 INFO  [M:0;master:61500] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@5af97169{/static,file:///opt/rills/software/apache-atlas-2.1.0/hbase/hbase-webapps/static/,UNAVAILABLE}
2023-06-15 09:06:30,547 INFO  [M:0;master:61500] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@57adfab0{/logs,file:///opt/rills/software/apache-atlas-2.1.0/hbase/logs/,UNAVAILABLE}
2023-06-15 09:06:30,549 INFO  [M:0;master:61500] regionserver.HRegionServer: stopping server master,61500,1686791181740
2023-06-15 09:06:30,549 INFO  [M:0;master:61500] zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x79f2feba to localhost:2181
2023-06-15 09:06:30,551 INFO  [ProcessThread(sid:0 cport:2181):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x188bc96d4a90002
2023-06-15 09:06:30,553 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn: Closed socket connection for client /0:0:0:0:0:0:0:1:55222 which had sessionid 0x188bc96d4a90002
2023-06-15 09:06:30,554 INFO  [ReadOnlyZKClient-localhost:2181@0x79f2feba] zookeeper.ZooKeeper: Session: 0x188bc96d4a90002 closed
2023-06-15 09:06:30,554 INFO  [ReadOnlyZKClient-localhost:2181@0x79f2feba-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x188bc96d4a90002
2023-06-15 09:06:30,558 INFO  [M:0;master:61500] regionserver.HRegionServer: stopping server master,61500,1686791181740; all regions closed.
2023-06-15 09:06:30,558 INFO  [M:0;master:61500] hbase.ChoreService: Chore service for: master/master:61500 had [] on shutdown
2023-06-15 09:06:30,559 WARN  [M:0;master:61500] master.ActiveMasterManager: Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null
2023-06-15 09:06:30,561 INFO  [M:0;master:61500] wal.WALProcedureStore: Stopping the WAL Procedure Store, isAbort=false
2023-06-15 09:06:30,562 INFO  [M:0;master:61500] hbase.ChoreService: Chore service for: master/master:61500.splitLogManager. had [] on shutdown
2023-06-15 09:06:30,563 INFO  [ProcessThread(sid:0 cport:2181):] server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x188bc96d4a90000 type:delete cxid:0x1a zxid:0x2a txntype:-1 reqpath:n/a Error Path:/hbase/rs/master,61500,1686791181740 Error:KeeperErrorCode = NoNode for /hbase/rs/master,61500,1686791181740
2023-06-15 09:06:30,565 INFO  [ProcessThread(sid:0 cport:2181):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x188bc96d4a90000
2023-06-15 09:06:30,567 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn: Closed socket connection for client /0:0:0:0:0:0:0:1:55218 which had sessionid 0x188bc96d4a90000
2023-06-15 09:06:30,568 INFO  [M:0;master:61500] zookeeper.ZooKeeper: Session: 0x188bc96d4a90000 closed
2023-06-15 09:06:30,568 INFO  [M:0;master:61500] regionserver.HRegionServer: Exiting; stopping=master,61500,1686791181740; zookeeper connection closed.
2023-06-15 09:06:30,568 INFO  [M:0;master:61500] server.NIOServerCnxn: Closed socket connection for client /127.0.0.1:47812 which had sessionid 0x188bc96d4a90001
2023-06-15 09:06:30,568 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x188bc96d4a90000
2023-06-15 09:06:30,568 INFO  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x188bc96d4a90001, likely server has closed socket, closing socket connection and attempting reconnect
2023-06-15 09:06:30,569 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: NIOServerCnxn factory exited run method
2023-06-15 09:06:30,572 INFO  [M:0;master:61500] server.ZooKeeperServer: shutting down
2023-06-15 09:06:30,572 ERROR [M:0;master:61500] server.ZooKeeperServer: ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
2023-06-15 09:06:30,572 INFO  [M:0;master:61500] server.SessionTrackerImpl: Shutting down
2023-06-15 09:06:30,572 INFO  [M:0;master:61500] server.PrepRequestProcessor: Shutting down
2023-06-15 09:06:30,572 INFO  [M:0;master:61500] server.SyncRequestProcessor: Shutting down
2023-06-15 09:06:30,572 INFO  [ProcessThread(sid:0 cport:2181):] server.PrepRequestProcessor: PrepRequestProcessor exited loop!
2023-06-15 09:06:30,572 INFO  [SyncThread:0] server.SyncRequestProcessor: SyncRequestProcessor exited!
2023-06-15 09:06:30,573 INFO  [M:0;master:61500] server.FinalRequestProcessor: shutdown of request processor complete
2023-06-15 09:06:30,574 INFO  [M:0;master:61500] zookeeper.MiniZooKeeperCluster: Shutdown MiniZK cluster with all ZK servers

在hbase-site.xml配置文件,添加如下配置:

<property><name>hbase.unsafe.stream.capability.enforce</name><value>false</value>
</property>

(三)ATLAS相关

1. ATLAS的日志在哪里?
/opt/rills/software/apache-atlas-2.1.0/logs/application.log
2. 内置HIVE的日志在哪里?
首先,日志存储在HIVE服务运行的机器上,在hive/conf/hive-log4j2.properties文件中进行了配置。
该文件如果不存在,就会使用默认存储路径,HIVE日志默认的存储路径是:
hive.root.logger=WARN,DRFAhive.log.dir=/tmp/${user.name}/${hive.log.file}(默认的文件名为:hive.log)
3. ATLAS启动非常慢?
ATLAS在第一次启动非常慢,一般都在10分钟左右,具体时间根据主机的性能而定。
如果ATLAS命令行启动显示成功 且 HBASE正常 且 SOLR正常,
那就只能等着ATLAS完全启动完毕后,UI才可以正常访问。

http://www.ppmy.cn/news/1524434.html

相关文章

从零开始学cv-0:图像处理基础知识

提示&#xff1a;文章写完后&#xff0c;目录可以自动生成&#xff0c;如何生成可参考右边的帮助文档 文章目录 前言一&#xff0c;图像分类1.1、模拟图像1.2、数字图像 二、颜色模式&#xff08;颜色存储&#xff09;2.1、RGB模式&#xff08;发光模式&#xff09;2.2、CMYK模…

pptp解说

PPTP&#xff08;Point-to-Point Tunneling Protocol&#xff09;是一种网络协议&#xff0c;用于在互联网上创建虚拟专用网络&#xff08;VPN&#xff09;连接。 PPTP 允许远程用户通过公共网络&#xff08;如互联网&#xff09;安全地连接到私有网络。它是一种较早的VPN技术…

鸿萌数据恢复服务:Mac 文件系统是如何影响 Mac 数据恢复的?

天津鸿萌科贸发展有限公司从事数据安全服务二十余年&#xff0c;致力于为各领域客户提供专业的数据备份、数据恢复解决方案与服务&#xff0c;并针对企业面临的数据安全风险&#xff0c;提供专业的相关数据安全培训。 公司是多款国际主流数据恢复软件的授权代理商&#xff0c;为…

C++设计模式——State状态模式

一&#xff0c;状态模式的定义 状态模式是一种行为型设计模式&#xff0c;状态模式允许对象在内部状态发生切换时改变它自身的行为。 状态模式的主要目的是将复杂的状态切换逻辑抽象化为一组离散的状态类&#xff0c;使代码结构更加清晰和易于维护。 状态模式将对象的行为封…

继收购西门子物流自动化后,丰田又投资一家AGV公司,智能物流版图已极其夸张...

导语 大家好&#xff0c;我是社长&#xff0c;老K。专注分享智能制造和智能仓储物流等内容。 继成功将西门子物流自动化(机场物流业务)纳入麾下后&#xff0c;丰田并未停下其征伐的步伐&#xff0c;而是再度出手&#xff0c;与新兴科技巨头Gideon携手&#xff0c;共同绘制了一幅…

AI 产品经理:2024 年职场新航标 ——AI 产品经理的未来与契机

前言 这两年&#xff0c;AI 骤然“火”了起来&#xff0c;可谓出现了重大“转折”。就在这短短两年间&#xff0c;全球各大“大厂”几乎在同一时间争先恐后地跟进 AI 技术。从 ChatGPT 发布起&#xff0c;谷歌、Facebook、亚马逊等纷纷紧跟其后&#xff0c;国内的百度、腾讯、…

【笔记】一维动态规划DP

文章目录 动态规划DPDP解题步骤例子1lanqiao3367 破损的楼梯题目描述输入格式输出格式解题思路代码 lanqiao3423 安全序列题目描述输入格式输出格式解题思路代码 动态规划DP 动态规划用于解决具有重叠子问题、最优子结构特征的问题。 重叠子问题&#xff1a;子问题是原问题的…

安装与pytorch不同cuda版本的bitsandbytes

由于历史遗留问题&#xff0c;虽然我的cuda版本是11.8&#xff0c;但是我的torch对应cuda版本是12.1&#xff0c;安装bitsandbytes后&#xff0c;就会抛出以下报错&#xff1a; Could not load bitsandbytes native library: libcusparse.so.12: cannot open shared object fi…