Nomadを動かしてみる

はじめに

Hashicorpが作っているNomadを試してみます。
マイクロサービスやbatch処理用に設計されたクラスタマネージャ・スケジューラで、dockerだけでなくbatch処理などにも利用でき、AWSだけでなくインフラのベンダーロックインしにくい特徴があります。

参考

https://www.hashicorp.com/blog/nomad.html

【参考訳】Nomad

テスト環境構築

[shell]
$ wget https://raw.githubusercontent.com/hashicorp/nomad/master/demo/vagrant/Vagrantfile
$ vagrant up
$ vagrant ssh
$ nomad
usage: nomad [–version] [–help] <command></command> []

Available commands are:
agent Runs a Nomad agent
agent-info Display status information about the local agent
alloc-status Display allocation status information and metadata
client-config View or modify client configuration details
eval-monitor Monitor an evaluation interactively
init Create an example job file
node-drain Toggle drain mode on a given node
node-status Display status information about nodes
run Run a new job or update an existing job
server-force-leave Force a server into the ‘left’ state
server-join Join server nodes together
server-members Display a list of known servers and their status
status Display status information about jobs
stop Stop a running job
validate Checks if a given job specification is valid
version Prints the Nomad version

[/shell]

nomad起動

[shell]

vagrant@nomad:~$ sudo nomad agent -dev
No configuration files loaded
==> Starting Nomad agent…
==> Nomad agent configuration:

Atlas:
Client: true
Log Level: DEBUG
Region: global (DC: dc1)
Server: true

==> Nomad agent started! Log data will stream in below:

2016/02/02 09:08:43 [INFO] serf: EventMemberJoin: nomad.global 127.0.0.1
2016/02/02 09:08:43 [INFO] nomad: starting 1 scheduling worker(s) for [service batch system _core]
2016/02/02 09:08:43 [INFO] client: using state directory /tmp/NomadClient833780201
ping
2016/02/02 09:08:47 [DEBUG] fingerprint.env_gce: Error querying GCE Metadata URL, skipping
2016/02/02 09:08:47 [DEBUG] fingerprint.network: Detected interface lo with IP 127.0.0.1 during fingerprinting
2016/02/02 09:08:47 [WARN] fingerprint.network: Unable to read link speed from /sys/class/net/lo/speed
2016/02/02 09:08:47 [DEBUG] fingerprint.network: Unable to read link speed; setting to default 100
2016/02/02 09:08:47 [DEBUG] client: applied fingerprints [arch cpu host memory network storage]
2016/02/02 09:08:47 [WARN] driver.raw_exec: raw exec is enabled. Only enable if needed
2016/02/02 09:08:47 [DEBUG] driver.docker: using client connection initialized from environment
2016/02/02 09:08:47 [DEBUG] driver.docker: privileged containers are disabled
2016/02/02 09:08:47 [DEBUG] client: available drivers [raw_exec docker exec]
2016/02/02 09:08:47 [DEBUG] client: node registration complete
2016/02/02 09:08:47 [DEBUG] client: updated allocations at index 1 (0 allocs)
2016/02/02 09:08:47 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 0)
2016/02/02 09:08:47 [DEBUG] client: state updated to ready

[/shell]

ステータスを取得

[shell]

vagrant@nomad:~$ nomad node-status
ID DC Name Class Drain Status
7772b8db-bdac-54ec-c67d-79b9e22ebe43 dc1 nomad false ready
vagrant@nomad:~$ nomad node-status
ID DC Name Class Drain Status
7772b8db-bdac-54ec-c67d-79b9e22ebe43 dc1 nomad false ready
[/shell]

メンバーを取得

[shell]
vagrant@nomad:~$ nomad server-members
Name Addr Port Status Proto Build DC Region
nomad.global 127.0.0.1 4648 alive 2 0.2.3 dc1 global
vagrant@nomad:~$ nomad server-members -detailed
Name Addr Port Tags
nomad.global 127.0.0.1 4648 bootstrap=1,build=0.2.3,port=4647,region=global,role=nomad,vsn_max=1,dc=dc1,vsn=1,vsn_min=1
[/shell]

サンプル設定ファイルを生成

[shell]
vagrant@nomad:~$ nomad init
Example job file written to example.nomad
vagrant@nomad:~$ ls
example.nomad
vagrant@nomad:~$ cat example.nomad
# There can only be a single job definition per file.
# Create a job with ID and Name ‘example’
job "example" {
# Run the job in the global region, which is the default.
# region = "global"

# Specify the datacenters within the region this job can run in.
datacenters = ["dc1"]

# Service type jobs optimize for long-lived services. This is
# the default but we can change to batch for short-lived tasks.
# type = "service"

# Priority controls our access to resources and scheduling priority.
# This can be 1 to 100, inclusively, and defaults to 50.
# priority = 50

# Restrict our job to only linux. We can specify multiple
# constraints as needed.
constraint {
attribute = "$attr.kernel.name"
value = "linux"
}

# Configure the job to do rolling updates
update {
# Stagger updates every 10 seconds
stagger = "10s"

# Update a single task at a time
max_parallel = 1
}

# Create a ‘cache’ group. Each task in the group will be
# scheduled onto the same machine.
group "cache" {
# Control the number of instances of this groups.
# Defaults to 1
# count = 1

# Restart Policy – This block defines the restart policy for TaskGroups,
# the attempts value defines the number of restarts Nomad will do if Tasks
# in this TaskGroup fails in a rolling window of interval duration
# The delay value makes Nomad wait for that duration to restart after a Task
# fails or crashes.
restart {
interval = "5m"
attempts = 10
delay = "25s"
}

# Define a task to run
task "redis" {
# Use Docker to run the task.
driver = "docker"

# Configure Docker driver with the image
config {
image = "redis:latest"
port_map {
db = 6379
}
}

service {
name = "${TASKGROUP}-redis"
tags = ["global", "cache"]
port = "db"
check {
name = "alive"
type = "tcp"
interval = "10s"
timeout = "2s"
}
}

# We must specify the resources required for
# this task to ensure it runs on a machine with
# enough capacity.
resources {
cpu = 500 # 500 Mhz
memory = 256 # 256MB
network {
mbits = 10
port "db" {
}
}
}
}
}
}vagrant@nomad:~$
[/shell]
redisが起動するようだ。

Runする

[shell]
vagrant@nomad:~$ nomad run example.nomad
==> Monitoring evaluation "a3316f59-b62f-57ef-30f5-b522bd25e418"
Evaluation triggered by job "example"
Allocation "19663bbf-cc8d-a045-c59a-859aa19e27d7" created: node "7772b8db-bdac-54ec-c67d-79b9e22ebe43", group "cache"
Evaluation status changed: "pending" -> "complete"
==> Evaluation "a3316f59-b62f-57ef-30f5-b522bd25e418" finished with status "complete"
vagrant@nomad:~$ ps axwww
PID TTY STAT TIME COMMAND
1 ? Ss 0:00 /sbin/init
2 ? S 0:00 [kthreadd]
3 ? S 0:00 [ksoftirqd/0]
5 ? S< 0:00 [kworker/0:0H]
7 ? S 0:00 [rcu_sched]
8 ? S 0:00 [rcu_bh]
9 ? S 0:00 [rcuos/0]
10 ? S 0:00 [rcuob/0]
11 ? S 0:00 [migration/0]
12 ? S 0:00 [watchdog/0]
13 ? S< 0:00 [khelper]
14 ? S 0:00 [kdevtmpfs]
15 ? S< 0:00 [netns]
16 ? S< 0:00 [perf]
17 ? S 0:00 [khungtaskd]
18 ? S< 0:00 [writeback]
19 ? SN 0:00 [ksmd]
20 ? SN 0:00 [khugepaged]
21 ? S< 0:00 [crypto]
22 ? S< 0:00 [kintegrityd]
23 ? S< 0:00 [bioset]
24 ? S< 0:00 [kblockd]
25 ? S< 0:00 [ata_sff]
26 ? S< 0:00 [md]
27 ? S< 0:00 [devfreq_wq]
28 ? S 0:00 [kworker/u2:1]
31 ? S 0:00 [kswapd0]
32 ? S 0:00 [fsnotify_mark]
33 ? S 0:00 [ecryptfs-kthrea]
45 ? S< 0:00 [kthrotld]
46 ? S< 0:00 [acpi_thermal_pm]
47 ? S 0:00 [scsi_eh_0]
48 ? S< 0:00 [scsi_tmf_0]
49 ? S 0:00 [scsi_eh_1]
50 ? S< 0:00 [scsi_tmf_1]
55 ? S< 0:00 [ipv6_addrconf]
75 ? S< 0:00 [deferwq]
76 ? S< 0:00 [charger_manager]
124 ? S< 0:00 [kpsmoused]
125 ? S 0:00 [kworker/0:2]
126 ? S 0:00 [scsi_eh_2]
128 ? S< 0:00 [scsi_tmf_2]
134 ? S< 0:00 [kdmflush]
135 ? S< 0:00 [bioset]
137 ? S< 0:00 [kdmflush]
140 ? S< 0:00 [bioset]
153 ? S 0:00 [jbd2/dm-0-8]
154 ? S< 0:00 [ext4-rsv-conver]
236 ? S< 0:00 [kworker/0:1H]
306 ? S 0:00 upstart-udev-bridge –daemon
310 ? Ss 0:00 /lib/systemd/systemd-udevd –daemon
334 ? S< 0:00 [ext4-rsv-conver]
376 ? S< 0:00 [rpciod]
382 ? Ss 0:00 dbus-daemon –system –fork
385 ? S< 0:00 [nfsiod]
402 ? Ss 0:00 rpc.idmapd
405 ? Ss 0:00 /lib/systemd/systemd-logind
409 ? S< 0:00 [iprt-VBoxWQueue]
411 ? Ssl 0:00 rsyslogd
434 ? S 0:00 upstart-file-bridge –daemon
497 ? Ss 0:00 rpcbind
542 ? Ss 0:00 rpc.statd -L
564 ? S 0:00 upstart-socket-bridge –daemon
768 tty4 Ss+ 0:00 /sbin/getty -8 38400 tty4
770 tty5 Ss+ 0:00 /sbin/getty -8 38400 tty5
774 tty2 Ss+ 0:00 /sbin/getty -8 38400 tty2
775 tty3 Ss+ 0:00 /sbin/getty -8 38400 tty3
777 tty6 Ss+ 0:00 /sbin/getty -8 38400 tty6
805 ? Ss 0:00 cron
846 ? Sl 0:07 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/mcollectived –pid=/var/run/puppetlabs/mcollectived.pid –config=/etc/puppetlabs/mcollective/server.cfg –daemonize
899 ? Ssl 0:02 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent
924 ? Sl 0:00 /opt/puppetlabs/puppet/bin/pxp-agent
1007 ? Sl 0:02 /usr/sbin/VBoxService –pidfile /var/run/vboxadd-service.pid
1079 tty1 Ss+ 0:00 /sbin/getty -8 38400 tty1
1090 ? S 0:00 [kauditd]
1439 ? Ss 0:00 dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0
1488 ? Ss 0:00 /usr/sbin/sshd -D
1895 ? Ss 0:00 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 103:109
1988 ? S 0:00 [kworker/u2:2]
3530 ? Ssl 0:10 /usr/bin/docker daemon
3570 ? S 0:02 [kworker/0:0]
3678 ? Ss 0:00 sshd: vagrant [priv]
3696 ? S 0:00 sshd: vagrant@pts/1
3697 pts/1 Ss 0:00 -bash
3997 pts/1 S+ 0:00 sudo nomad agent -dev
3998 pts/1 Sl+ 0:03 nomad agent -dev
4028 ? Ss 0:00 sshd: vagrant [priv]
4046 ? R 0:00 sshd: vagrant@pts/2
4047 pts/2 Ss 0:00 -bash
4300 ? Sl 0:00 docker-proxy -proto tcp -host-ip 127.0.0.1 -host-port 48211 -container-ip 172.17.0.2 -container-port 6379
4308 ? Sl 0:00 docker-proxy -proto udp -host-ip 127.0.0.1 -host-port 48211 -container-ip 172.17.0.2 -container-port 6379
4314 ? Ssl 0:00 redis-server *:6379
4336 pts/2 R+ 0:00 ps axwww
[/shell]
redis-serverが起動している。

ステータス確認

[shell]
vagrant@nomad:~$ nomad status
ID Type Priority Status
example service 50 <none>
vagrant@nomad:~$ nomad status example
ID = example
Name = example
Type = service
Priority = 50
Datacenters = dc1
Status = <none>

==> Evaluations
ID Priority TriggeredBy Status
a3316f59-b62f-57ef-30f5-b522bd25e418 50 job-register complete

==> Allocations
ID EvalID NodeID TaskGroup Desired Status
19663bbf-cc8d-a045-c59a-859aa19e27d7 a3316f59-b62f-57ef-30f5-b522bd25e418 7772b8db-bdac-54ec-c67d-79b9e22ebe43 cache run running
[/shell]
起動しているNodeIDなどがわかる。

Stopする

[shell]
vagrant@nomad:~$ nomad stop example
==> Monitoring evaluation "ba629ece-851d-c161-802d-e522a297d171"
Evaluation triggered by job "example"
Evaluation status changed: "pending" -> "complete"
==> Evaluation "ba629ece-851d-c161-802d-e522a297d171" finished with status "complete"
vagrant@nomad:~$ nomad status example
Error querying job: Unexpected response code: 404 (job not found)
vagrant@nomad:~$ ps axwww | grep redis
4400 pts/2 S+ 0:00 grep –color=auto redis
[/shell]
redis-serverがstopしている。