> For the complete documentation index, see [llms.txt](https://tech.x2bee.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://tech.x2bee.com/dev-guide/pjt-prepare/undefined/kubernetics.md).

# 서비스(kubernetics) 구성 설계

### Application Service

<figure><img src="/files/xNGuSKWwkIyzPNNoiewn" alt=""><figcaption></figcaption></figure>

<table data-header-hidden><thead><tr><th width="268.4444580078125"></th><th></th></tr></thead><tbody><tr><td><strong>리소스 구성</strong></td><td><strong>정보</strong></td></tr><tr><td><strong>Client / BFF / GW – BO, CC</strong></td><td><ul><li>BO와 CC는 클라이언트(PC 브라우저 + JavaScript)를 대상으로 하며, Server Side Rendering(SSR)을 지원하는 Next.js로 어플리케이션아 구성된다.</li></ul></td></tr><tr><td><strong>Client / BFF / GW – FO (PC, MO)</strong></td><td><ul><li>FO(PC, MO) Client는 Next.js로 개발되며 FO Gateway로 부터 Restful API 서비스를 제공받는다.</li><li>위 그림은 논리적 구성을 표현하며 실제적으로는 Node.js를 이용한 SSR을 한다.</li></ul></td></tr><tr><td><strong>APIs</strong></td><td><ul><li>Springboot로 구현된 Restful API 서비스</li><li>order, member, common, goods, display, event와 bo API 서비스로 구성된다.</li></ul></td></tr><tr><td><strong>DBMS</strong></td><td><ul><li>회원/주문 {order, member, common}</li><li>상품/전시{goods, display}</li><li>이벤트 {event}</li><li>Master(Read-Write)/Replica(Read Only) 구조로 이중화</li><li>ETL 혹은 배치잡을 이용한 table 동기화</li></ul></td></tr></tbody></table>

### Log Stack

<figure><img src="/files/eSDGM3GqJts1wUT9vg4E" alt=""><figcaption></figcaption></figure>

<table data-header-hidden><thead><tr><th width="145.111083984375"></th><th></th></tr></thead><tbody><tr><td><strong>리소스 구성</strong></td><td><strong>정보</strong></td></tr><tr><td><strong>OpenSearch</strong></td><td><ul><li>수집되는 application 로그를 인덱싱하여 저장</li></ul></td></tr><tr><td><strong>Fluentd</strong></td><td><ul><li>Application의 로그를 수집하여 포매팅하여 OpenSearch에 전송</li></ul></td></tr><tr><td><strong>Kibana</strong></td><td><ul><li>OpenSearch의 로그를 개발자가 조회할 수 있는 UI 제공</li></ul></td></tr><tr><td><strong>EFK 서버 위치</strong></td><td><ul><li>fluentd : Kubernetes</li><li>OpenSearch : AWS(Managed Service)</li><li>Kibana : AWS(Managed Service)</li></ul></td></tr><tr><td><strong>로그 보관</strong></td><td><ul><li>OpenSearch와는 별개로 로그는 보관 기간에 따라 S3에 압축 보관</li></ul></td></tr></tbody></table>

### Telemetry

<figure><img src="/files/JGt78vabItxlpJ0sj09h" alt=""><figcaption></figcaption></figure>

<table data-header-hidden><thead><tr><th width="194.0001220703125"></th><th></th></tr></thead><tbody><tr><td><strong>리소스 구성</strong></td><td><strong>정보</strong></td></tr><tr><td><strong>APM</strong></td><td><ul><li>어플리케이션 성능 관리. Request 처리 성능의 병목 구간을 찾아 개선</li><li>APM제공 라이브러리를 함께 application에 구성<br>(* 사용 제품 미정)</li></ul></td></tr><tr><td><strong>Prometheous-Grafana</strong></td><td><ul><li>Pod에 metric(cpu, memory, network utilization 등) 모니터링</li><li>Istio envoy에서 제공</li></ul></td></tr><tr><td><strong>Kiali</strong></td><td><ul><li>MSA 간 호출의 처리량, 응답속도, 정상여부 상황을 트래픽 flow로 모니터링</li><li>Istio envoy에서 제공</li></ul></td></tr><tr><td><strong>Jaeger</strong></td><td><ul><li>하나의 request 처리를 위한 MSA간의 호출, 처리 관계를 상세 분석할 수 있도록 기능 제공</li><li>Istio envoy에서 제공</li></ul></td></tr></tbody></table>

### Software 구성

<figure><img src="/files/q36rZXXsuNCDSzjvmlpO" alt=""><figcaption></figcaption></figure>

<table data-header-hidden><thead><tr><th width="178.4444580078125"></th><th></th></tr></thead><tbody><tr><td><strong>리소스 구성</strong></td><td><strong>정보</strong></td></tr><tr><td><strong>Application</strong></td><td>Springboot</td></tr><tr><td><strong>Container</strong></td><td>Docker</td></tr><tr><td><strong>Orchestration</strong></td><td>EKS(Kubernetes)</td></tr><tr><td><strong>Servismesh</strong></td><td>istio</td></tr><tr><td><strong>CICD</strong></td><td>Gitlab, Nexus, Jenkins, Sonarqube, Helm Chart, ArgoCD</td></tr><tr><td><strong>Logging</strong></td><td>EFK (OpenSearch, Fluentd, Kibana)</td></tr><tr><td><strong>Telemetry</strong></td><td>EKS</td></tr><tr><td><strong>Batch Scheduler</strong></td><td>Cronicle</td></tr><tr><td><strong>APM</strong></td><td>WhaTap or Scouter</td></tr><tr><td><strong>DBMS</strong></td><td>PostgreSQL</td></tr><tr><td><strong>In-Memory</strong></td><td>Redis(Session Clustering)</td></tr></tbody></table>

### Software 목록

| **적용 영역**                     | **적용 솔루션**     | **라이선스**               | **버전 / 기타**               |
| ----------------------------- | -------------- | ---------------------- | ------------------------- |
| \*\* External Proxy (ALB)\*\* | AWS ALB        | AWS PaaS               | Managed Service           |
| **Servie Mesh (G/W)**         | Istio          | Apache License 2.0     | Version 1.8 이상            |
| **Orchastration**             | Kubernetes     | Apache License 2.0     | Version 1.19 이상           |
| **Data Storage**              | S3(AWS)        | AWS PaaS               | Managed Service           |
| **CI/CD**                     | Gitlab         | MIT License            | 소스 통합                     |
|                               | Jenkins        | MIT License            | CI/CD Pipeline            |
|                               | ArogCD         | Apache License 2.0     | Kubernetes Deployment     |
|                               | Helm           | Apache License 2.0     | Kubernetes Deployment     |
|                               | NexusOSS       | Eclipse Public License | Library, Docker Image 저장소 |
| **Logging**                   | Fluentd        | Apache License 2.0     | Application 로그 수집         |
|                               | OpenSearch     | AWS PaaS               | Managed Service, 로그 인덱스   |
|                               | Kibana         | AWS PaaS               | Managed Service, 로그 분석 화면 |
| **Telemetry**                 | Prometheus     | Apache License 2.0     | 서버 metric 수집              |
|                               | Grafana        | Apache License 2.0     | 서버 metric UI 제공           |
|                               | Kiali          | Apache License 2.0     | API 호출 모니터링               |
|                               | Jaeger(Zipkin) | Apache License 2.0     | API 호출 상세 분석              |
|                               | WhaTap (미정)    | 유료 라이선스                | TP Monitor (미정)           |
| **DBMS**                      | PostgreSQL     | OSI License            | Persistence DBMS          |
| **In-Memory DB**              | Redis          | AWS PaaS               | Managed Service, 세션 클러스터링 |

### Kubernetes Apps

<figure><img src="/files/z6BIU9iL42n4cCYuZNtk" alt=""><figcaption></figcaption></figure>

| **Deployment / Pod**           | **Namespace** | **Workload** | **Node Label** | **Description**                                           |
| ------------------------------ | ------------- | ------------ | -------------- | --------------------------------------------------------- |
| **x2bee-fo**                   | \[고객사약어]-app  | deployment   | msa            | Node.js Service for PC Web(Next.js Server Side Rendering) |
| **x2bee-gw**                   | \[고객사약어]-app  | deployment   | msa            | RestAPIs Service for Vue.js (Spring Cloud Gateway)        |
| **x2bee-bo**                   | \[고객사약어]-app  | deployment   | msa            | Back Office Web Service (Spring Boot, Next.js)            |
| **x2bee-cc**                   | \[고객사약어]-app  | deployment   | msa            | Customer Center Web Service (Spring Boot, Next.js)        |
| **x2bee-batch-mbod**           | \[고객사약어]-app  | deployment   | msa            | Batch service for member, order (Spring Batch)            |
| **x2bee-batch-gddp**           | \[고객사약어]-app  | deployment   | msa            | Batch service for goods, display (Spring Batch)           |
| **x2bee-api-member**           | \[고객사약어]-app  | deployment   | msa            | Member api service (Springboot, RestApis)                 |
| **x2bee-api-order**            | \[고객사약어]-app  | deployment   | msa            | Order api service (Springboot, RestApis)                  |
| **x2bee-api-goods**            | \[고객사약어]-app  | deployment   | msa            | Goods api service (Springboot, RestApis)                  |
| **x2bee-api-display**          | \[고객사약어]-app  | deployment   | msa            | Display api service (Springboot, RestApis)                |
| **x2bee-api-event**            | \[고객사약어]-app  | deployment   | msa            | Event api service (Springboot, RestApis)                  |
| **x2bee-api-common**           | \[고객사약어]-app  | deployment   | msa            | Common api service (Springboot, RestApis)                 |
| **x2bee-api-bo**               | \[고객사약어]-app  | deployment   | msa            | Bo api service (Springboot, RestApis)                     |
| **istio-ingressgateway**       | istio-system  | deployment   | msa            | Istio ingress component                                   |
| **istio-tracing**              | istio-system  | deployment   | msa            | istio tracing component                                   |
| **x2bee-batch-scheduler-mbod** | \[고객사약어]-app  | deployment   | msa            | Batch scheduler for batch-mbod (node.js)                  |
| **x2bee-batch-scheduler-gddp** | \[고객사약어]-app  | deployment   | msa            | Batch scheduler for batch-gddp (node.js)                  |
| **grafana**                    | monitoring    | deployment   | msa            | monitoring bi                                             |
| **prometheus**                 | monitoring    | deployment   | msa            | to gather pod metrics                                     |
| **kiali**                      | istio-system  | deployment   | msa            | to monitor msa calls flows                                |
| **jaeger**                     | istio-system  | deployment   | msa            | to trace each msa call chain                              |