[docker study] load balancer -> nginx -> deno -> mongo 까지 연결 테스트

[Backend study] - [docker study] 오라클 로드밸런서 <--> nginx <--> deno server

이전에 오라클 클라우드에서 로드 밸런서에서 deno 웹 서버까지 연결 테스트 했다. mongo를 docker 켄테이너로 설치하고 deno 서버에서 mongo 서버로 connect까지 해 본다. 

50GB volume을 /mnt/djkim 에 마운트했다.

이미 오라클 vm에 자체 block volume을 붙여놓았다. 50GB 용량을 가지고 있고, 이것은 어떤 VM에도 attach될 수 있다. 나중에 vm이 업그레이드 될 경우에도 block volume을 붙이면 데이타 그대로 유지된다. 현재는 mongo db의 저장소로 사용할 예정이다. 

자세한 내용은  https://thekoguryo.github.io/oci/chapter03/6/3/ 를 참고했다.   

[연결할 명령어는 오래클 클라우드 vm의 볼륨 페이지의 ... 버튼에 있음]

[opc@instance-20221029-2034 ~]$ sudo iscsiadm -m node -o new -T iqn.2015-12.com.oracleiaas:46c16679-d0e5-4258-bb65-24260665245e -p 169.254.2.2:3260
New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 169.254.2.2,3260,-1 iqn.2015-12.com.oracleiaas:46c16679-d0e5-4258-bb65-24260665245e] added
[opc@instance-20221029-2034 ~]$ sudo iscsiadm -m node -o update -T iqn.2015-12.com.oracleiaas:46c16679-d0e5-4258-bb65-24260665245e -n node.startup -v automatic
[opc@instance-20221029-2034 ~]$ sudo iscsiadm -m node -T iqn.2015-12.com.oracleiaas:46c16679-d0e5-4258-bb65-24260665245e -p 169.254.2.2:3260 -l
Logging in to [iface: default, target: iqn.2015-12.com.oracleiaas:46c16679-d0e5-4258-bb65-24260665245e, portal: 169.254.2.2,3260]
Login to [iface: default, target: iqn.2015-12.com.oracleiaas:46c16679-d0e5-4258-bb65-24260665245e, portal: 169.254.2.2,3260] successful.



[Disk /dev/sdb: 50 GiB, 확인]
[opc@instance-20221029-2034 ~]$ sudo fdisk -l
Disk /dev/sda: 46.6 GiB, 50010783744 bytes, 97677312 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disklabel type: gpt
Disk identifier: 347FC70E-7421-4C2C-82F1-CCEF19B84F66

Device       Start      End  Sectors  Size Type
/dev/sda1     2048   206847   204800  100M EFI System
/dev/sda2   206848  2303999  2097152    1G Linux filesystem
/dev/sda3  2304000 97675263 95371264 45.5G Linux LVM


Disk /dev/mapper/ocivolume-root: 35.5 GiB, 38088474624 bytes, 74391552 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes


Disk /dev/mapper/ocivolume-oled: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes


Disk /dev/sdb: 50 GiB, 53687091200 bytes, 104857600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes




[디스크 포맷할 파일포맷 확인]
[opc@instance-20221029-2034 ~]$ lsblk -f
NAME               FSTYPE      LABEL UUID                                   MOUNTPOINT
sda
|-sda1             vfat              3A04-27E1                              /boot/efi
|-sda2             xfs               92f46bb4-0609-4149-8155-521cb271344d   /boot
`-sda3             LVM2_member       eu3JyT-prH1-YQe4-5zOP-LDd1-fcTk-8XGRod
  |-ocivolume-root xfs               a2c46b56-da9f-4829-be87-1d87b18c8aeb   /
  `-ocivolume-oled xfs               f9d1a191-470c-4bb2-ad65-58f895357a5d   /var/oled
sdb



[포맷]
[opc@instance-20221029-2034 ~]$ sudo mkfs -t xfs /dev/sdb
meta-data=/dev/sdb               isize=512    agcount=4, agsize=3276800 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1
data     =                       bsize=4096   blocks=13107200, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=25600, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0


[포맷 확인]
[opc@instance-20221029-2034 ~]$ lsblk -f
NAME               FSTYPE      LABEL UUID                                   MOUNTPOINT
sda
|-sda1             vfat              3A04-27E1                              /boot/efi
|-sda2             xfs               92f46bb4-0609-4149-8155-521cb271344d   /boot
`-sda3             LVM2_member       eu3JyT-prH1-YQe4-5zOP-LDd1-fcTk-8XGRod
  |-ocivolume-root xfs               a2c46b56-da9f-4829-be87-1d87b18c8aeb   /
  `-ocivolume-oled xfs               f9d1a191-470c-4bb2-ad65-58f895357a5d   /var/oled
sdb                xfs               4a4f9fdc-1111-43ed-90a6-edc3d552aa08



[마운트할 디렉토리 생성]
[opc@instance-20221029-2034 ~]$ sudo mkdir /mnt/djkim



[/etc/fstab에 마지막 한 줄 추가]
opc@instance-20221029-2034 ~]$ sudo vi /etc/fstab 
/dev/oracleoci/oraclevdb /mnt/djkim xfs defaults,_netdev,nofail 0 2



[마운트하기]
[opc@instance-20221029-2034 ~]$ sudo mount -a



[디렉토리 확인]
[opc@instance-20221029-2034 djkim]$ ls -al /mnt/djkim
total 0
drwxr-xr-x. 2 root root  6 Nov 17 15:08 .
drwxr-xr-x. 3 root root 19 Nov 17 15:14 ..

 

mongo  서버 구동

mongo/docker-compose.yml

version: '3.1'

services:

  mongo:
    image: mongo:6.0.2
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: example
    volumes:
      - /mnt/djkim/mongo-storage:/data/db
      - ./:/app
    working_dir: /app
    hostname: mongo-host
    ports:
      - 27017:27017
    networks:
      - backend


networks:
  backend:
    driver: bridge
    external: true

내가 붙인 50GB에 /mnt/djkim/mongo-storage에 mongo 서버 데이타 디렉토리로 지정한다.  호스트 이름은 mongo-host 로 지정한다.  현재 디렉토리도 /app 으로 연결한다.  

실행과 확인

[opc@instance-20221029-2034 mongo]$ docker-compose up -d
Recreating mongo_mongo_1 ... done
[opc@instance-20221029-2034 mongo]$ sudo docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS         PORTS                                           NAMES
eccd20973e1f   mongo:6.0.2            "docker-entrypoint.s…"   10 seconds ago   Up 6 seconds   0.0.0.0:27017->27017/tcp, :::27017->27017/tcp   mongo_mongo_1
c9a0b542887a   nginx:1.21.6           "/docker-entrypoint.…"   2 days ago       Up 2 days      0.0.0.0:80->80/tcp, :::80->80/tcp               nginx_nginx_1
42b92c017ed8   denoland/deno:1.28.0   "/tini -- docker-ent…"   2 days ago       Up 2 days      0.0.0.0:8080->8080/tcp, :::8080->8080/tcp       deno_service_deno_1
[opc@instance-20221029-2034 mongo]$ sudo docker exec -it mongo_mongo_1 bash
root@mongo-host:/app# pwd
/app

 

deno 서버에서 Mongo 연결

이전 포스트의 deno_service/app.ts 수정.. 딱히 하는 일은 없고 db 서버에 연결만 해본다. 

import { serve } from "https://deno.land/std@0.164.0/http/server.ts";
import { Bson, MongoClient } from "https://deno.land/x/mongo/mod.ts"

class MyMongoClient {
    dbName: string
    url: string
    client: MongoClient

    constructor(dbName: string, url: string) {
        this.dbName = dbName;
        this.url = url;
        this.client = {} as MongoClient;
    }

    async connect() {
        console.log("mongodb connecting ...")
        const client = new MongoClient();
        await client.connect(this.url);
        this.client = client;
        console.log("mongodb connected OK")
    }

    getDatabase() {
        return this.client.database(this.dbName);
    }
}


export const mongoClient = new MyMongoClient("test-db", "mongodb://root:example@mongo-host:27017")
await mongoClient.connect()


const port = 8080;

const handler = (request: Request): Response => {
  const body = `Your user-agent is:\n\n${
    request.headers.get("user-agent") ?? "Unknown"
  }`;

  return new Response(body, { status: 200 });
};

console.log(`HTTP webserver running. Access it at: http://localhost:8080/`);
await serve(handler, { port });

실행 확인

[opc@instance-20221029-2034 deno_service]$ docker-compose up -d
Starting deno_service_deno_1 ... done
[opc@instance-20221029-2034 deno_service]$ sudo docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS                                           NAMES
eccd20973e1f   mongo:6.0.2            "docker-entrypoint.s…"   19 minutes ago   Up 18 minutes   0.0.0.0:27017->27017/tcp, :::27017->27017/tcp   mongo_mongo_1
c9a0b542887a   nginx:1.21.6           "/docker-entrypoint.…"   2 days ago       Up 2 days       0.0.0.0:80->80/tcp, :::80->80/tcp               nginx_nginx_1
42b92c017ed8   denoland/deno:1.28.0   "/tini -- docker-ent…"   2 days ago       Up 12 seconds   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp       deno_service_deno_1

[opc@instance-20221029-2034 deno_service]$ sudo docker logs --tail 10 deno_service_deno_1
Download https://deno.land/x/web_bson@v0.2.5/src/validate_utf8.ts
Download https://deno.land/x/web_bson@v0.2.5/src/float_parser.ts
Download https://deno.land/x/mongo@v0.31.1/src/utils/saslprep/deps.ts
Download https://deno.land/x/mongo@v0.31.1/src/utils/saslprep/load_code_points.ts
Download https://deno.land/x/mongo@v0.31.1/src/utils/saslprep/sparse_bitfield.ts
Download https://deno.land/x/mongo@v0.31.1/src/utils/saslprep/memory_pager.ts
mongodb connecting ...
mongodb connected OK
HTTP webserver running. Access it at: http://localhost:8080/
Listening on http://localhost:8080/

mongodb connected OK가 찍힌걸로 봐서 연결까지 된 듯 하다. 

mongodb://root:example@mongo-host:27017 로 연결은 된 듯 하지만 실제로 읽기나 쓰기를 하면 auth 에러가 날 것이다.  test-db에 권한이 없다고 나올 텐데 권한을 추가하자..  https://dajkim76.tistory.com/560 참고

[opc@instance-20221029-2034 deno_service]$ docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS                                           NAMES
eccd20973e1f   mongo:6.0.2            "docker-entrypoint.s…"   46 minutes ago   Up 46 minutes   0.0.0.0:27017->27017/tcp, :::27017->27017/tcp   mongo_mongo_1
c9a0b542887a   nginx:1.21.6           "/docker-entrypoint.…"   2 days ago       Up 2 days       0.0.0.0:80->80/tcp, :::80->80/tcp               nginx_nginx_1
42b92c017ed8   denoland/deno:1.28.0   "/tini -- docker-ent…"   2 days ago       Up 27 minutes   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp       deno_service_deno_1
[opc@instance-20221029-2034 deno_service]$ docker exec -it mongo_mongo_1 bash

root@mongo-host:/app# mongosh -u root -p example
Current Mongosh Log ID:	63765dee3755cc7187c2c247
Connecting to:		mongodb://<credentials>@127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.6.0
Using MongoDB:		6.0.2
Using Mongosh:		1.6.0

For mongosh info see: https://docs.mongodb.com/mongodb-shell/

------
   The server generated these startup warnings when booting
   2022-11-17T15:25:14.257+00:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'
   2022-11-17T15:25:14.257+00:00: vm.max_map_count is too low
------

------
   Enable MongoDB's free cloud-based monitoring service, which will then receive and display
   metrics about your deployment (disk utilization, CPU, operation statistics, etc).

   The monitoring data will be available on a MongoDB website with a unique URL accessible to you
   and anyone you share the URL with. MongoDB may use this information to make product
   improvements and to suggest MongoDB products and deployment options to you.

   To enable free monitoring, run the following command: db.enableFreeMonitoring()
   To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
------

test> use test-db
switched to db test-db
test-db> db.createUser( { user: "testuser", pwd: "testpwd", roles: [ { role: "readWrite", db: "test-db" }, ] } )
{ ok: 1 }
test-db>

이제 mongodb://testuser:testpwd@mongo-host:27017 로 다시 연결을 만들고 read/write를 해보면 문제 없다.  테스트 해보지는 않았지만 아마도 될 듯.

테스트해 보자.. mongosh로 데이타를  하나 넣었다..

test> use test-db
switched to db test-db
test-db> db.user_list.insertOne({name:"name1", age:31})
{
  acknowledged: true,
  insertedId: ObjectId("63766d72b310e7e91006854b")
}
test-db>

app.ts 다시 수정. 쓸데없는 코드 다 빼고 핵심만 보이도록 수정했다. https://deno.land/x/mongo@v0.31.1/tests/cases/02_connect.ts?source. 참고

import { serve } from "https://deno.land/std@0.164.0/http/server.ts";
import { Bson, MongoClient } from "https://deno.land/x/mongo/mod.ts"

export const mongoClient = new MongoClient();
await mongoClient.connect("mongodb://testuser:testpwd@mongo-host:27017/test-db")

interface UserSchema {
  _id: ObjectId;
  name: string;
  age: number;
}


const userCollection = mongoClient.database().collection<UserSchema>("user_list");
const cursor = userCollection.find();
const usersResult = await cursor.toArray();
const json = JSON.stringify(usersResult);
console.log(json);


const port = 8080;

const handler = (request: Request): Response => {
  const body = `Your user-agent is:\n${json}\n${
    request.headers.get("user-agent") ?? "Unknown"
  }`;

  return new Response(body, { status: 200 });
};

console.log(`HTTP webserver running. Access it at: http://localhost:8080/`);
await serve(handler, { port });

실행
[opc@instance-20221029-2034 deno_service]$ docker-compose up -d
Starting deno_service_deno_1 ... done

결과

dajkim76@Kims-Mac-mini ~ %  curl https://oci.mdiwebma.com/deno-app
Your user-agent is:
[{"_id":"63766d72b310e7e91006854b","name":"name1","age":31}]
curl/7.79.1

https://oci.mdiwebma.com/deno-app. 누르면 오라클 클라우드 로드밸랜서 -> VM Nginx -> Deno app 서버 -> mongo db 순으로 요청 흐름이 발생하고 test-db의 user_list 콜렉션을 전부 가져와서 응답한다.  nginx, deno, mongo는 모두 docker container로 동작한다.