Skip to content

phenix workflow

Every command on this page also accepts the global flags.

phenix workflow

Workflow management

Synopsis

Workflow management

  Deploy topology directories through the running phenix server over its
  unix socket (--unix-socket), and print their configs with the variables
  filled in.
phenix workflow [flags]

Options

  -h, --help   help for workflow

SEE ALSO

phenix workflow apply

Deploy a topology directory through the phenix server

Synopsis

Deploy a topology directory through the running phenix server

  A topology directory holds any of:

    phenix-injects/   copied to <base-dir.injects>/<name>
    phenix-configs/   configs upserted to the server, Topology configs first;
                      only .json, .yaml and .yml files count, and hidden
                      files are skipped
    phenix.yml        the workflow config (any other name, such as
                      phenix.yaml, needs -c)
    .env              variables for the configs, as NAME=value lines

  DIR|NAME is required. An existing path is used as is. Otherwise a NAME
  with no path separator is looked up under --base-dir.topologies. The
  branch name defaults to the base name of the directory; -b sets
  another. -b, --branch-name and --name are the same flag. Give only one of
  them, or give them the same value.

  The branch name is not the experiment name. The experiment name is
  auto.create in the workflow config. With "auto.create: ${BRANCH_NAME}",
  -b also names the experiment, and each branch name is a separate
  deployment. A fixed auto.create ignores -b. phenix reads auto.create
  only when no experiment is mapped to the branch.

  apply fills in the ${NAME} and ${NAME:default} placeholders of each
  config and of the workflow config to check them. Then it sends each
  file as written, with the variables, and the server fills them in, also
  in each topology file that includeTopologies names by a path. A value comes
  from -e, else from an env file, else from the shell. --env-file names
  the env files; without it, apply reads .env in the topology directory.
  The shell gives no PHENIX_ variable unless -e NAME names it. A
  placeholder with no value and no default is an error, and so is a -e
  variable that no file uses. With a workflow config, a topology file that
  a topology includes by a path counts as a file. Without a workflow
  config, it does not. Write
  ${NAME:} for an optional variable and $${NAME} for the literal text
  ${NAME}. Only -b sets BRANCH_NAME, and the server fills in
  ${BRANCH_NAME}. Files in phenix-injects/ are not changed.

  The names phenix.yml, phenix-configs and phenix-injects are
  reserved, in any letter case: they cannot be the branch name,
  and no entry of phenix-injects/ may have one of them at its top level
  or directly inside one of its top-level directories.

  apply talks to the server over --unix-socket and changes nothing until
  step 3. Each log record names its step in the step field:

    1. preflight: check the server, fill in the variables, and check that
       every file is one YAML or JSON document and that every config has
       a kind
    2. plan: the server dry-runs every config, then the workflow config
       with those configs as pending, and checks what they refer to;
       every rejection is logged, any rejection, or two files that
       define the same config, stops the apply, and the plan is logged
    3. injects: copy phenix-injects/ to the injects directory the server
       reports, or to --base-dir.injects when it is given on the command
       line
    4. configs: upsert every config; every rejection is logged, and any
       rejection stops the apply
    5. apply: apply the workflow config, tagged with where it came from

  A missing input skips its step. -n stops after step 2. A plan that
  would restart a running experiment needs -f. Progress is logged at the
  info level, so --log.level warn hides it. Skipped steps, and an apply
  that changes no experiment (action none), are logged as warnings.
phenix workflow apply <DIR|NAME> [flags]

Examples

# Deploy by path; the branch name is the directory name, helloworld
phenix workflow apply /phenix/topologies/helloworld

# Look helloworld up under --base-dir.topologies; works from any directory
phenix workflow apply helloworld

# Deploy the current directory
phenix workflow apply .

# Validate and log the plan; change nothing
phenix workflow apply helloworld -n

# Allow restarting the running experiment
phenix workflow apply helloworld -f

# Deploy under another branch name; injects go to <base-dir.injects>/helloworld-2
phenix workflow apply helloworld -b helloworld-2

# The same; --branch-name and --name are the same as -b
phenix workflow apply helloworld --name helloworld-2

# Set variables for the ${NAME} placeholders; -e overrides an env file
phenix workflow apply helloworld -e MEMORY=4096 -e SITE=lab

# Copy MEMORY from the shell; it then overrides helloworld/.env
phenix workflow apply helloworld -e MEMORY

# Read the env files, and not helloworld/.env; site.env overrides base.env
phenix workflow apply helloworld --env-file base.env --env-file site.env

# Follow symlinks in phenix-injects/ and copy targets as regular files and directories
phenix workflow apply helloworld --follow-inject-symlinks

# Use another workflow config
phenix workflow apply helloworld -c alt.yml

# Dry run of an alternate config under another branch name
phenix workflow apply helloworld -b hw-test -c alt.yml -n

# A directory with only phenix-configs/: upsert the configs; injects and apply are skipped
phenix workflow apply ./topo

# Use a non-default socket
phenix --unix-socket /run/phenix.sock workflow apply helloworld

# Use another topology root
phenix --base-dir.topologies /srv/topologies workflow apply helloworld

# The same, from the environment
PHENIX_BASE_DIR_TOPOLOGIES=/srv/topologies phenix workflow apply helloworld

# Override the injects directory the server reports
phenix --base-dir.injects /srv/injects workflow apply helloworld

# Log request and response detail
phenix --log.level debug workflow apply helloworld -n

# Compose deployment; the socket is inside the container
docker exec phenix phenix workflow apply /phenix/topologies/helloworld -f

Options

  -b, --branch-name string       workflow branch name (default: base name of the topology directory); it does not rename an experiment
  -c, --config string            workflow config, relative to the topology directory unless absolute (default "phenix.yml")
  -n, --dry-run                  check every config and the workflow config, log the server's plan, and change nothing
  -e, --env stringArray          set a variable for the ${NAME} placeholders: NAME=value, or NAME to copy the value from the shell (repeatable)
      --env-file stringArray     read variables from an env file of NAME=value lines; a relative path starts at the working directory (repeatable)
      --follow-inject-symlinks   follow symlinks in phenix-injects/ and copy targets as regular files and directories
  -f, --force                    allow apply to stop, reconfigure and restart a running experiment
  -h, --help                     help for apply
      --name string              same as -b/--branch-name

SEE ALSO

phenix workflow render

Print the configs of a topology directory with their variables filled in

Synopsis

Print the configs of a topology directory with their variables filled in

  render reads a topology directory as apply does: the configs in
  phenix-configs/, the workflow config, and the variables from -e, the env
  files and the shell. It fills in each ${NAME} and ${NAME:default}
  placeholder, ${BRANCH_NAME} included, and prints the text of each file as
  the server fills it in and then stores or applies it. It does not contact the
  server, and it changes nothing. It does not read the topology files that
  includeTopologies names, so a -e variable that only such a file uses is
  an error in render.

  The output holds each config in upsert order, Topology configs first, and
  then the workflow config. A comment line with the path of the file comes
  before each file, and a line with --- comes between two files. render
  does not format the text of a file again.

  A problem with a variable is an error, as it is in apply. The output can
  hold secrets: render prints the value of each variable that a file uses.
phenix workflow render <DIR|NAME> [flags]

Examples

# Print the configs of helloworld with their variables filled in
phenix workflow render helloworld

# Show what another branch name and a variable give
phenix workflow render helloworld -b helloworld-2 -e MEMORY=4096

# Use an env file and another workflow config
phenix workflow render ./topo --env-file site.env -c alt.yml

Options

  -b, --branch-name string     workflow branch name (default: base name of the topology directory); it does not rename an experiment
  -c, --config string          workflow config, relative to the topology directory unless absolute (default "phenix.yml")
  -e, --env stringArray        set a variable for the ${NAME} placeholders: NAME=value, or NAME to copy the value from the shell (repeatable)
      --env-file stringArray   read variables from an env file of NAME=value lines; a relative path starts at the working directory (repeatable)
  -h, --help                   help for render
      --name string            same as -b/--branch-name

SEE ALSO