02-01---flink+clickhouse构建亿级电商实时数据平台

虚拟机的语句详解：
1.zookeeper
启动：zkServer.sh start 
zkServer.sh start 
zkServer.sh status
zkCli.sh
2.kafka
kafka-server-start.sh /usr/local/apps/kafka/config/server.properties 
kafka如何创建topic：<a href="https://blog.csdn.net/lanmolei814/article/details/78353898">https://blog.csdn.net/lanmolei814/article/details/78353898</a>
3.flink操作语句
进入cd /usr/local/apps/flink/bin
启动flink:./start-cluster.sh 
4.hadoop安装
hadoop namenode -format
start-dfs.sh start-yarn.sh
5.clickhouse
service clickhouse-server start service clickhouse-server stop service clickhouse-server restart
clickhouse--client
6.hbase
start-hbase.sh
代码的github地址：<a href="https://github.com/FandySww/FlinkAnanlySystem">https://github.com/FandySww/FlinkAnanlySystem</a>
资料：
<img alt="" height="283" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-d3ac62e2832838cf645e3da9ac622364.png" width="642">
---
数据收集的日志结构：
<img alt="" height="325" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-e204a47dba00ea36aa0ffc46ad47297d.png" width="533">
小程序：
<img alt="" height="313" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-69279e70341e5e979bca5885f9a223c8.png" width="361">
app的：
<img alt="" height="365" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-2a1a12f22c9acd3cc27fc23b56d40ad4.png" width="482">
搜集我们的浏览日志：
<img alt="" height="377" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-8e2fc2e491434618490b2fc222892ed3.png" width="309">
我们把这些结构变成实体，方便我们去处理。
<img alt="" height="130" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-665fd8481ab6a0cd547478399c11744d.png" width="304">
---1-5---
代码：
<img alt="" height="141" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-1a0c0ae40115144a58ff3c0f5e655374.png" width="377">
fastjson和null的做法：
<img alt="" height="319" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-ff526929d1e079d487baea4bbd8012b0.png" width="492">
<img alt="" height="684" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-5e048ec172217ac44ed705bca6e9339e.png" width="770">
---1-6---
kafka环境的搭建：
资料：
<img alt="" height="235" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-16c66dee7ce57477b2e2062cb1cc6304.png" width="815">
centos虚拟机的克隆：<a href="https://blog.csdn.net/qq_28764557/article/details/88851553">https://blog.csdn.net/qq_28764557/article/details/88851553</a>
虚拟机的地址： 133 134 135
<img alt="" height="89" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-f04ff59201fdb5b0c494b394e12352fe.png" width="183">
安装zk和kafka。
在131上安装了zookeeper和kafka。
zk单机安装：
第一步修改环境变量，具体略。
第二步修配置文件：
<img alt="" height="206" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-5f89ffa88015c25128533e3f58e5fba3.png" width="550">
<img alt="" height="275" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-512ebe837e3e0ba4cc52abfd44dd9c7b.png" width="462">
---
kafka的安装，第一步，直接配置
<img alt="" height="132" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-3fef604c2e854ead0bc91915b4f78361.png" width="335">
第二步：两个地方：
1.<img alt="" height="119" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-3ede30807659b20cd0fe2db2bf379526.png" width="333">
2.<img alt="" height="95" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-5a46d768a59c6ce780549cd2c6484a60.png" width="358">
3.<img alt="" height="394" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-1eacf98b3c1fdf00fa9ac69e8478bd8b.png" width="857">
第三步：配置环境变量，略
第四步：启动
<img alt="" height="133" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-fc454fefa584ac6726e3564674d20fa2.png" width="782">
kafka如何创建topic:在bin目录下./kafka-topics.sh --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 1 --topic dataiinfo
---
启动总结：
zk：
<img alt="" height="335" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-73da1d840b8b8a3c57ae7b3567c46372.png" width="349">
kafka：
<img alt="" height="153" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-47a4ffd26c66e24b631b14bd3e839d61.png" width="676">
---1-7---
回到代码：
<img alt="" height="234" src="https://beijingoptbbs.oss-cn-beijing.aliyuncs.com/cs/5606289-28f0558f528e41174985b94eb3bb0e13.png" width="628">
向topic发消息：./kafk