phenix workflow¶
Every command on this page also accepts the global flags.
phenix workflow¶
Workflow management
Synopsis¶
Workflow management
Deploy topology directories through the running phenix server over its
unix socket (--unix-socket), and print their configs with the variables
filled in.
Options¶
SEE ALSO¶
- phenix - A cli application for phēnix
- phenix workflow apply - Deploy a topology directory through the phenix server
- phenix workflow render - Print the configs of a topology directory with their variables filled in
phenix workflow apply¶
Deploy a topology directory through the phenix server
Synopsis¶
Deploy a topology directory through the running phenix server
A topology directory holds any of:
phenix-injects/ copied to <base-dir.injects>/<name>
phenix-configs/ configs upserted to the server, Topology configs first;
only .json, .yaml and .yml files count, and hidden
files are skipped
phenix.yml the workflow config (any other name, such as
phenix.yaml, needs -c)
.env variables for the configs, as NAME=value lines
DIR|NAME is required. An existing path is used as is. Otherwise a NAME
with no path separator is looked up under --base-dir.topologies. The
branch name defaults to the base name of the directory; -b sets
another. -b, --branch-name and --name are the same flag. Give only one of
them, or give them the same value.
The branch name is not the experiment name. The experiment name is
auto.create in the workflow config. With "auto.create: ${BRANCH_NAME}",
-b also names the experiment, and each branch name is a separate
deployment. A fixed auto.create ignores -b. phenix reads auto.create
only when no experiment is mapped to the branch.
apply fills in the ${NAME} and ${NAME:default} placeholders of each
config and of the workflow config to check them. Then it sends each
file as written, with the variables, and the server fills them in, also
in each topology file that includeTopologies names by a path. A value comes
from -e, else from an env file, else from the shell. --env-file names
the env files; without it, apply reads .env in the topology directory.
The shell gives no PHENIX_ variable unless -e NAME names it. A
placeholder with no value and no default is an error, and so is a -e
variable that no file uses. With a workflow config, a topology file that
a topology includes by a path counts as a file. Without a workflow
config, it does not. Write
${NAME:} for an optional variable and $${NAME} for the literal text
${NAME}. Only -b sets BRANCH_NAME, and the server fills in
${BRANCH_NAME}. Files in phenix-injects/ are not changed.
The names phenix.yml, phenix-configs and phenix-injects are
reserved, in any letter case: they cannot be the branch name,
and no entry of phenix-injects/ may have one of them at its top level
or directly inside one of its top-level directories.
apply talks to the server over --unix-socket and changes nothing until
step 3. Each log record names its step in the step field:
1. preflight: check the server, fill in the variables, and check that
every file is one YAML or JSON document and that every config has
a kind
2. plan: the server dry-runs every config, then the workflow config
with those configs as pending, and checks what they refer to;
every rejection is logged, any rejection, or two files that
define the same config, stops the apply, and the plan is logged
3. injects: copy phenix-injects/ to the injects directory the server
reports, or to --base-dir.injects when it is given on the command
line
4. configs: upsert every config; every rejection is logged, and any
rejection stops the apply
5. apply: apply the workflow config, tagged with where it came from
A missing input skips its step. -n stops after step 2. A plan that
would restart a running experiment needs -f. Progress is logged at the
info level, so --log.level warn hides it. Skipped steps, and an apply
that changes no experiment (action none), are logged as warnings.
Examples¶
# Deploy by path; the branch name is the directory name, helloworld
phenix workflow apply /phenix/topologies/helloworld
# Look helloworld up under --base-dir.topologies; works from any directory
phenix workflow apply helloworld
# Deploy the current directory
phenix workflow apply .
# Validate and log the plan; change nothing
phenix workflow apply helloworld -n
# Allow restarting the running experiment
phenix workflow apply helloworld -f
# Deploy under another branch name; injects go to <base-dir.injects>/helloworld-2
phenix workflow apply helloworld -b helloworld-2
# The same; --branch-name and --name are the same as -b
phenix workflow apply helloworld --name helloworld-2
# Set variables for the ${NAME} placeholders; -e overrides an env file
phenix workflow apply helloworld -e MEMORY=4096 -e SITE=lab
# Copy MEMORY from the shell; it then overrides helloworld/.env
phenix workflow apply helloworld -e MEMORY
# Read the env files, and not helloworld/.env; site.env overrides base.env
phenix workflow apply helloworld --env-file base.env --env-file site.env
# Follow symlinks in phenix-injects/ and copy targets as regular files and directories
phenix workflow apply helloworld --follow-inject-symlinks
# Use another workflow config
phenix workflow apply helloworld -c alt.yml
# Dry run of an alternate config under another branch name
phenix workflow apply helloworld -b hw-test -c alt.yml -n
# A directory with only phenix-configs/: upsert the configs; injects and apply are skipped
phenix workflow apply ./topo
# Use a non-default socket
phenix --unix-socket /run/phenix.sock workflow apply helloworld
# Use another topology root
phenix --base-dir.topologies /srv/topologies workflow apply helloworld
# The same, from the environment
PHENIX_BASE_DIR_TOPOLOGIES=/srv/topologies phenix workflow apply helloworld
# Override the injects directory the server reports
phenix --base-dir.injects /srv/injects workflow apply helloworld
# Log request and response detail
phenix --log.level debug workflow apply helloworld -n
# Compose deployment; the socket is inside the container
docker exec phenix phenix workflow apply /phenix/topologies/helloworld -f
Options¶
-b, --branch-name string workflow branch name (default: base name of the topology directory); it does not rename an experiment
-c, --config string workflow config, relative to the topology directory unless absolute (default "phenix.yml")
-n, --dry-run check every config and the workflow config, log the server's plan, and change nothing
-e, --env stringArray set a variable for the ${NAME} placeholders: NAME=value, or NAME to copy the value from the shell (repeatable)
--env-file stringArray read variables from an env file of NAME=value lines; a relative path starts at the working directory (repeatable)
--follow-inject-symlinks follow symlinks in phenix-injects/ and copy targets as regular files and directories
-f, --force allow apply to stop, reconfigure and restart a running experiment
-h, --help help for apply
--name string same as -b/--branch-name
SEE ALSO¶
- phenix workflow - Workflow management
phenix workflow render¶
Print the configs of a topology directory with their variables filled in
Synopsis¶
Print the configs of a topology directory with their variables filled in
render reads a topology directory as apply does: the configs in
phenix-configs/, the workflow config, and the variables from -e, the env
files and the shell. It fills in each ${NAME} and ${NAME:default}
placeholder, ${BRANCH_NAME} included, and prints the text of each file as
the server fills it in and then stores or applies it. It does not contact the
server, and it changes nothing. It does not read the topology files that
includeTopologies names, so a -e variable that only such a file uses is
an error in render.
The output holds each config in upsert order, Topology configs first, and
then the workflow config. A comment line with the path of the file comes
before each file, and a line with --- comes between two files. render
does not format the text of a file again.
A problem with a variable is an error, as it is in apply. The output can
hold secrets: render prints the value of each variable that a file uses.
Examples¶
# Print the configs of helloworld with their variables filled in
phenix workflow render helloworld
# Show what another branch name and a variable give
phenix workflow render helloworld -b helloworld-2 -e MEMORY=4096
# Use an env file and another workflow config
phenix workflow render ./topo --env-file site.env -c alt.yml
Options¶
-b, --branch-name string workflow branch name (default: base name of the topology directory); it does not rename an experiment
-c, --config string workflow config, relative to the topology directory unless absolute (default "phenix.yml")
-e, --env stringArray set a variable for the ${NAME} placeholders: NAME=value, or NAME to copy the value from the shell (repeatable)
--env-file stringArray read variables from an env file of NAME=value lines; a relative path starts at the working directory (repeatable)
-h, --help help for render
--name string same as -b/--branch-name
SEE ALSO¶
- phenix workflow - Workflow management