Compare commits

...

635 Commits

Author SHA1 Message Date
Colin Adler 84ec2dc1db chore: update git to v2.43.4
Fixes 3 CVEs recently reported:
  - [CVE-2024-32002](https://avd.aquasec.com/nvd/2024/cve-2024-32002/)
  - [CVE-2024-32004](https://avd.aquasec.com/nvd/2024/cve-2024-32004/)
  - [CVE-2024-32465](https://avd.aquasec.com/nvd/2024/cve-2024-32465/)
2024-05-22 19:36:30 +00:00
Stephen Kirby a11b169029 v2.10.2 changelog 2024-04-22 20:27:37 +00:00
Kyle Carberry 2a98123701 chore: add generate script for azure instance identity (#13028)
* chore: add generate script for azure instance identity

This also adds new issuing certificates from:
https://learn.microsoft.com/en-us/azure/security/fundamentals/azure-ca-details?tabs=certificate-authority-chains

* Fix shell lint

* Fix shell fmt

* Fix RSA issuing certificate
2024-04-22 19:43:21 +00:00
Stephen Kirby 2ed7226e85 added mainline disclaimer 2024-04-17 22:38:41 +00:00
Stephen Kirby 2101dbce03 updated version flags in kube install 2024-04-17 22:35:02 +00:00
Stephen Kirby cdeba67944 v2.10.1 changelog 2024-04-17 22:33:20 +00:00
Dean Sheather bda13a2818 fix: make terminal raw in ssh command on windows (#12990)
(cherry picked from commit d426569d4a)
2024-04-17 21:01:58 +00:00
Spike Curtis 353888a5d8 feat: add src_id and dst_id indexes to tailnet_tunnels (#12911)
Fixes #12780

Adds indexes to the `tailnet_tunnels` table to speed up `GetTailnetTunnelPeerIDs` and `GetTailnetTunnelPeerBindings` queries, which match on `src_id` and `dst_id`.

(cherry picked from commit a231b5aef5)
2024-04-17 21:01:39 +00:00
Spike Curtis 3fc6111994 fix: stop sending DeleteTailnetPeer when coordinator is unhealthy (#12925)
fixes #12923

Prevents Coordinate peer connections from generating spurious database queries like DeleteTailnetPeer when the coordinator is unhealthy.

It does this by checking the health of the querier before accepting a connection, rather than unconditionally accepting it only for it to get swatted down later.

(cherry picked from commit 06eae954c9)
2024-04-17 21:01:24 +00:00
Colin Adler 3eb9abcbd3 fix(coderd): prevent agent reverse proxy from using HTTP[S]_PROXY envs (#12875)
Updates https://github.com/coder/coder/issues/12790

(cherry picked from commit a2b28f80d7)
2024-04-17 21:01:12 +00:00
Stephen Kirby a7234f61a1 chore: update change log to v2.10.0 and install docs for release channels (#12863)
* 2.10.0 changelog

* updated install docs for mainline/stable releases

* make fmt

* cpp icon -> C++

* added disclaimer on MAX_TTL, support bundle info

* 'release schedule'

* lowercase mainline

* Agent OOM protection info

* minor tweak
2024-04-03 16:43:49 -05:00
Stephen Kirby bf19e3469f added 'JFrog' in front of XRay in guide (#12860) 2024-04-03 14:44:57 -05:00
Stephen Kirby d9211b6693 chore(docs): replace FAQ twisties with h3s (#12859)
* replace FAQ twisties with h3s

* make fmt
2024-04-03 14:44:28 -05:00
Colin Adler cb6fea61df chore: upgrade go to 1.21.9 (#12861) 2024-04-03 13:20:26 -05:00
Steven Masley a3187dc30f chore: enforce unique linked_ids (#12815)
* chore: enforce unique linked_ids

Duplicate linked_ids make no sense. 2 users cannot share the same
source user from a provider
2024-04-03 13:17:11 -05:00
Mathias Fredriksson 65f8d18ce5 feat(install.sh): add support for --mainline (default) and --stable (#12858)
Fixes #12461
2024-04-03 20:26:48 +03:00
Jon Ayers 426e9f2b96 feat: support adjusting child proc oom scores (#12655) 2024-04-03 09:42:03 -05:00
Marcin Tojek ac8d1c6696 docs: describe single region and multi-region deployments (#12779) 2024-04-03 12:45:01 +02:00
Kayla Washburn-Love caa49ea6a1 chore: stabilize light theme (#12855) 2024-04-02 16:06:31 -06:00
Colin Adler 41914256b3 chore: update terraform version in install.sh (#12856) 2024-04-02 16:53:36 -05:00
Kayla Washburn-Love 1dd840d149 test: add an e2e test for removing a group (#12844) 2024-04-02 11:29:43 -06:00
Kayla Washburn-Love f705f9a5eb test: ensure RequireActiveVersion is actually set when testing with AGPL store (#12843) 2024-04-02 11:29:22 -06:00
Kyle Carberry 7698cfda72 chore: remove unnecessary extraction library (#12847)
This was allocating ~256KB on init.
2024-04-02 11:19:54 -04:00
Steven Masley b5b5c37d03 docs: describe mutually exclusive create workspace template fields (#12834)
* docs: describe mutually exclusive create workspace template fields

Ideally we could do this in the OpenAPI spec, but there is no first
class "mutually exclusive" feature in OpenAPI. So in lieu of something
more complex, or changing our struct/validation, a description comment
should suffice.

* chore: Add description to code sample as well
2024-04-02 10:11:24 -05:00
Steven Masley 5137433123 chore: add validation errors to the cli output (#12814)
* chore: add validation errors to the cli output
2024-04-02 10:02:30 -05:00
Kyle Carberry 94e82f9662 chore: use fork of chroma to remove unused inits (#12842)
* chore: use fork of chroma to remove unused inits

This seems fine to do since compilation errors would occur
if it were actually in use.

Everything seems fine here.

* Update validator
2024-04-02 14:14:38 +00:00
Danny Kopping 79fb8e43c5 feat: expose workspace statuses (with details) as a prometheus metric (#12762)
Implements #12462
2024-04-02 09:57:36 +02:00
Toshiki Shimomura 114830de26 Fix coder-logstream-kube typo in deployment-logs.md (#12845) 2024-04-02 03:36:35 +00:00
Kyle Carberry f5a70500d2 chore: update tailscale for to lazily load hostinfo (#12840)
Speeds up `init()` by ~10ms
2024-04-01 17:04:39 -04:00
dependabot[bot] b47fb41783 chore: bump github.com/bramvdbogaerde/go-scp from 1.3.0 to 1.4.0 (#12825)
Bumps [github.com/bramvdbogaerde/go-scp](https://github.com/bramvdbogaerde/go-scp) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/bramvdbogaerde/go-scp/releases)
- [Commits](https://github.com/bramvdbogaerde/go-scp/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/bramvdbogaerde/go-scp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 20:48:52 +00:00
Kyle Carberry 2a30194ed7 chore: use validator fork to fix 10ms of init time (#12837)
@ammario inspired me
2024-04-01 16:31:26 -04:00
Kyle Carberry d428c05694 chore: move log output message before logs begin streaming (#12836) 2024-04-01 20:02:50 +00:00
Bruno Quaresma 7c1d10b952 chore(site): upgrade storybook to v8 (#12831) 2024-04-01 16:12:17 -03:00
Spike Curtis 3addf7ac5d fix: use latest coder/tailscale
Bad merge in #12252 clobbered #12574
2024-04-01 22:42:38 +04:00
dependabot[bot] 12ecc6554c chore: bump google.golang.org/api from 0.171.0 to 0.172.0 (#12827)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.171.0 to 0.172.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.171.0...v0.172.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 18:14:17 +00:00
dependabot[bot] a4be2831d6 chore: bump github.com/charmbracelet/glamour from 0.6.0 to 0.7.0 (#12824)
Bumps [github.com/charmbracelet/glamour](https://github.com/charmbracelet/glamour) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/charmbracelet/glamour/releases)
- [Commits](https://github.com/charmbracelet/glamour/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: github.com/charmbracelet/glamour
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 18:13:51 +00:00
dependabot[bot] 3e28250849 chore: bump github.com/cenkalti/backoff/v4 from 4.2.1 to 4.3.0 (#12826)
Bumps [github.com/cenkalti/backoff/v4](https://github.com/cenkalti/backoff) from 4.2.1 to 4.3.0.
- [Commits](https://github.com/cenkalti/backoff/compare/v4.2.1...v4.3.0)

---
updated-dependencies:
- dependency-name: github.com/cenkalti/backoff/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 13:02:23 -05:00
dependabot[bot] b7f5456e35 ci: bump the github-actions group with 1 update (#12828)
Bumps the github-actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action).


Updates `aquasecurity/trivy-action` from 0.18.0 to 0.19.0
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](https://github.com/aquasecurity/trivy-action/compare/062f2592684a31eb3aa050cc61e7ca1451cecd3d...d710430a6722f083d3b36b8339ff66b32f22ee55)

---
updated-dependencies:
- dependency-name: aquasecurity/trivy-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 13:01:09 -05:00
Bruno Quaresma 2f437005b7 chore(site): clean up mocks after each test (#12805) 2024-04-01 13:14:36 -03:00
elasticspoon cfb94284e0 feat(cli): add golden tests for errors (#11588) (#12698)
* feat(cli): add golden tests for errors (#11588)

Creates golden files from `coder/cli/errors.go`.
Adds a unit test to test against golden files.
Adds a make file command to regenerate golden files.
Abstracts test against golden files.
2024-04-01 09:19:26 -05:00
Bruno Quaresma 75bf41ba02 chore(site): use static date and ignore dynamic values for storybook (#12830) 2024-04-01 11:02:54 -03:00
Kayla Washburn-Love 1d2d008b45 chore: add e2e tests for template permissions (#12731) 2024-03-29 10:43:23 -06:00
Steven Masley eeb3d63be6 chore: merge authorization contexts (#12816)
* chore: merge authorization contexts

Instead of 2 auth contexts from apikey and dbauthz, merge them to
just use dbauthz. It is annoying to have two.

* fixup authorization reference
2024-03-29 10:14:27 -05:00
Michael Brewer 8e2d026d99 docs: document how to run workspace-proxy as a system service (#12810)
* docs: document how to run workspace-proxy as a system service

* Update workspace-proxies.md

* Update workspace-proxies.md

Co-authored-by: Muhammad Atif Ali <me@matifali.dev>

* docs: fix duplication

---------

Co-authored-by: Muhammad Atif Ali <me@matifali.dev>
2024-03-29 15:06:32 +00:00
Kayla Washburn-Love f3cfe10c26 chore: add more e2e template settings tests (#12717) 2024-03-28 16:00:27 -06:00
Steven Masley 8cf1e84bb5 chore: ensure root handler has sudomain app mw (#12812)
Enterprise routes like scim touch this.
2024-03-28 15:49:43 -05:00
Steven Masley b785e996f8 chore: explain GIT_ASKPASS behavior in docs (#12784)
* chore: docs explaining GIT_ASKPASS behavior

- VSCode configuration requirements
2024-03-28 13:59:03 -05:00
Mathias Fredriksson 79441e3609 perf(coderd/database): optimize GetWorkspaceAgentAndLatestBuildByAuthToken (#12809) 2024-03-28 19:38:16 +02:00
Steven Masley 93a233ac10 chore: write auto-update message after success (#12804) 2024-03-28 08:55:15 -05:00
Mathias Fredriksson d50c20c453 fix(coderd/database): add fk index for workspace_agent_scripts (#12791) 2024-03-28 14:31:58 +02:00
Danny Kopping d734f3fb74 chore: reduce azure CA cert validity check period to 2 months (#12788)
Signed-off-by: Danny Kopping <danny@coder.com>
2024-03-28 11:17:02 +02:00
Muhammad Atif Ali 0288e73e9b docs: add guide for Xray integration (#12629)
* docs: add guides for Xray integration

* `make fmt`
2024-03-28 04:48:50 +03:00
Colin Adler dc8cf3eea5 fix: nil ptr dereference when removing a license (#12785) 2024-03-27 15:59:35 -05:00
Muhammad Atif Ali 5235faa79f chore(site): remove max ttl from scheduling description (#12715) 2024-03-27 14:24:26 -05:00
Mathias Fredriksson 539d6b0f3b test(coderd): fix template name too long in TestPatchTemplateMeta (#12781) 2024-03-27 18:25:42 +02:00
Mathias Fredriksson 421bf7e785 fix(coderd): use insights for DAUs, simplify metricscache (#12775)
Fixes #12134
Fixes https://github.com/coder/customers/issues/384
Refs #12122
2024-03-27 18:10:14 +02:00
Bruno Quaresma 5d82a78d4c fix(site): fix and improve pending state on template editor UI (#12766) 2024-03-27 12:42:07 -03:00
Mathias Fredriksson 47fd190064 fix(coderd/database): improve perf of GetTemplateInsightsByInterval (#12773)
Refs #12122
2024-03-27 14:10:46 +02:00
Mathias Fredriksson ba1eaceda4 feat(coderd): add sftp to insights apps (#12675) 2024-03-27 14:09:29 +02:00
Danny Kopping 6cb1fc8956 chore: add note about options use in numeric parameters (#12770) 2024-03-27 13:46:41 +02:00
Mathias Fredriksson 0da29d74ac fix(coderd/database): improve query perf of GetTemplateAppInsights (#12767)
Refs #12122
2024-03-27 12:28:36 +02:00
Danny Kopping a74ef4096e feat: allow number options with monotonic validation (#12726)
NOTE: terraform-provider-coder was updated to facilitate this change, and your template will require v0.19.0 for this feature to work. You can run terraform init -upgrade in your template directory. If you have a version constraint set, ensure it points to this version.
2024-03-27 08:54:42 +00:00
Ammar Bandukwala 0d9010e150 chore: fix 30% startup time hit from userpassword (#12769)
pbkdf2 is too expensive to run in init, so this change makes it load
lazily. I introduced a lazy package that I hope to use more in my
`GODEBUG=inittrace=1` adventure.


Benchmark results:

```
$ hyperfine "coder --help" "coder-new --help"
Benchmark 1: coder --help
  Time (mean ± σ):      82.1 ms ±   3.8 ms    [User: 93.3 ms, System: 30.4 ms]
  Range (min … max):    72.2 ms …  90.7 ms    35 runs
 
Benchmark 2: coder-new --help
  Time (mean ± σ):      52.0 ms ±   4.3 ms    [User: 62.4 ms, System: 30.8 ms]
  Range (min … max):    41.9 ms …  62.2 ms    52 runs
 
Summary
  coder-new --help ran
    1.58 ± 0.15 times faster than coder --help
```
2024-03-26 20:47:14 -05:00
dependabot[bot] 73fbdbbe2d chore: bump github.com/coreos/go-oidc/v3 from 3.9.0 to 3.10.0 (#12739)
Bumps [github.com/coreos/go-oidc/v3](https://github.com/coreos/go-oidc) from 3.9.0 to 3.10.0.
- [Release notes](https://github.com/coreos/go-oidc/releases)
- [Commits](https://github.com/coreos/go-oidc/compare/v3.9.0...v3.10.0)

---
updated-dependencies:
- dependency-name: github.com/coreos/go-oidc/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-26 12:45:42 -05:00
Colin Adler 4d5a7b2d56 chore(codersdk): move all tailscale imports out of codersdk (#12735)
Currently, importing `codersdk` just to interact with the API requires
importing tailscale, which causes builds to fail unless manually using
our fork.
2024-03-26 12:44:31 -05:00
dependabot[bot] 0bea8906d4 chore: bump google.golang.org/api from 0.170.0 to 0.171.0 (#12737)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.170.0 to 0.171.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.170.0...v0.171.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-26 17:14:01 +00:00
dependabot[bot] a323e30450 chore: bump github.com/aws/aws-sdk-go-v2 from 1.25.3 to 1.26.0 (#12738)
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.25.3 to 1.26.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.25.3...v1.26.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-26 12:03:01 -05:00
Mathias Fredriksson ae0ee622bb fix(coderd/database): improve data exclusion in UpsertTemplateUsageStats (#12764)
The PostgreSQL query analyzer wasn't able to eliminate the agent stats without re-introducing this filter.

Before: https://explain.dalibo.com/plan/21h7gb4f4bef391g
After: https://explain.dalibo.com/plan/721ec1cccee91egc
2024-03-26 17:21:05 +02:00
Mathias Fredriksson f418ece9ae test(cli): prevent flake due to outdated build in TestSSH (#12760)
Fixes #12752
2024-03-26 10:46:58 +00:00
Spike Curtis 51491fc01b fix(provisionersdk): change test to use bash script instead of binary echo (#12759)
Just upgraded to macOS 14.4 and TestAgentScript/Run fails for me with error `signal: killed`.  I opened the test directory in a terminal and sure enough, when you execute the `echo` binary, it is immediately killed.  The binary has no extended attributes and is byte-identical to the one in `/bin/`.

This fix uses a different strategy: instead of copying the `echo` binary from the system around, we just copy a small bash script that _calls_ the `echo` command.
2024-03-26 14:37:20 +04:00
Colin Adler 5f28220eec fix(coderd): add timeout to websocket waitgroup on shutdown (#12754) 2024-03-26 03:04:15 +00:00
Kayla Washburn-Love cfb484fa25 fix: always use bash when executing web terminal tests (#12755) 2024-03-25 16:58:07 -06:00
dependabot[bot] 064a08efa5 ci: bump the github-actions group with 1 update (#12743)
Bumps the github-actions group with 1 update: [contributor-assistant/github-action](https://github.com/contributor-assistant/github-action).


Updates `contributor-assistant/github-action` from 2.3.1 to 2.3.2
- [Release notes](https://github.com/contributor-assistant/github-action/releases)
- [Commits](https://github.com/contributor-assistant/github-action/compare/v2.3.1...v2.3.2)

---
updated-dependencies:
- dependency-name: contributor-assistant/github-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-25 23:40:35 +03:00
Asher 40e5ad5499 feat: make OAuth2 provider not enterprise-only (#12732) 2024-03-25 11:52:22 -08:00
Garrett Delfosse 60f335113c chore: add protocol lag to shared ports description (#12728) 2024-03-25 15:28:38 -04:00
Kyle Carberry fd8010c26d chore: make build only run on main (#12753) 2024-03-25 19:15:18 +00:00
Kayla Washburn-Love 541ccd940c chore: change e2e testing port (#12751) 2024-03-25 13:07:34 -06:00
Kyle Carberry 03ab37b343 chore: remove middleware to request version and entitlement warnings (#12750)
This cleans up `root.go` a bit, adds tests for middleware HTTP transport
functions, and removes two HTTP requests we always always performed previously
when executing *any* client command.

It should improve CLI performance (especially for users with higher latency).
2024-03-25 15:01:42 -04:00
Cian Johnston ba3879ac47 fix(cli): fix newline escape sequence in support blurb (#12749) 2024-03-25 16:51:48 +00:00
Marcin Tojek 1e0bbd5e10 docs: describe operational readiness (#12723) 2024-03-25 17:10:24 +01:00
Mathias Fredriksson 7e183db199 test(coderd): fix todo for increased accuracy in insights test (#12727)
This PR updates the tests in `insights_test.go` to enable commented-out scenarios. This behavior was fixed by previous PRs in this stack. Note that the updated golden files are correct since they are "second template only" meaning that the newly introduced data is considered as expected. In other golden files there is no change since "only count once" is applied.
2024-03-25 17:55:53 +02:00
Mathias Fredriksson b183236482 feat(coderd/database): use template_usage_stats in *ByTemplate insights queries (#12668)
This PR updates the `*ByTempalte` insights queries used for generating Prometheus metrics to behave the same way as the new rollup query and re-written insights queries that utilize the rolled up data.
2024-03-25 17:42:02 +02:00
Cian Johnston 01f9a9ab77 feat(cli): unhide support bundle cmd (#12745)
* chore(cli): add another test to ensure no secret leakage

* feat(cli): unhide support bundle cmd
2024-03-25 15:14:27 +00:00
Mathias Fredriksson 2332d8197a feat(coderd/database): use template_usage_stats in GetUserActivityInsights query (#12672)
This PR updates the `GetUserActivityInsights` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 16:16:41 +02:00
Mathias Fredriksson a8ed689bda feat(coderd/database): use template_usage_stats in GetUserLatencyInsights query (#12671)
This PR updates the `GetUserLatencyInsights` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 16:07:40 +02:00
Mathias Fredriksson 5738a03930 feat(coderd/database): use template_usage_stats in GetTemplateAppInsights query (#12669)
This PR updates the `GetTemplateAppInsights` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 15:58:37 +02:00
Mathias Fredriksson 5f3be62c83 feat(coderd/database): use template_usage_stats in GetTemplateInsightsByInterval query (#12667)
This PR updates the `GetTemplateInsightsByInterval` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 15:45:49 +02:00
Mathias Fredriksson 35d08434a9 feat(coderd/database): use template_usage_stats in GetTemplateInsights query (#12666)
This PR updates the `GetTemplateInsights` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 15:33:31 +02:00
Mathias Fredriksson f34592f45d fix(coderd): skip logging error for cancelled query in agent report stats (#12730) 2024-03-25 12:20:16 +02:00
Jon Ayers 951dfaa99c feat: add workspace_id to workspace_build audit logs (#12718) 2024-03-22 15:10:32 -05:00
Kayla Washburn-Love 0966fe2560 fix: create workspace with optional auth providers (#12729) 2024-03-22 13:26:02 -06:00
Steven Masley c674128105 chore: allow search by build params in workspace search filter (#12694)
* chore: workspace search filter allow search by params
* has_param will return all workspaces with the param existance
* exact matching
2024-03-22 14:22:47 -05:00
Mathias Fredriksson b4fd819f0d test(coderd): enable dbrollup service for insights tests (#12673) 2024-03-22 20:18:20 +02:00
Mathias Fredriksson 12e6fbf11e feat(coderd/database): add dbrollup service to rollup insights (#12665)
Add `dbrollup` service that runs the `UpsertTemplateUsageStats` query
every 5 minutes, on the minute. This allows us to have fairly real-time
insights data when viewing "today".
2024-03-22 18:42:43 +02:00
Mathias Fredriksson 04f0510b09 feat(coderd/database): add template_usage_stats table and rollup query (#12664)
Add `template_usage_stats` table for aggregating tempalte usage data.
Data is rolled up by the `UpsertTemplateUsageStats` query, which fetches
data from the `workspace_agent_stats` and `workspace_app_stats` tables.
2024-03-22 18:33:34 +02:00
Kyle Carberry a6b8f381f0 feat: support windows containers in bootstrap script (#12662)
Fixes #7462.
2024-03-22 14:48:51 +00:00
Muhammad Atif Ali 58cbd8335f chore(site): reorganize template schedule strings page (#12714) 2024-03-22 15:44:16 +03:00
Marcin Tojek a7d9d87ba2 docs: use scale testing utility (#12643) 2024-03-22 11:33:31 +00:00
Colin Adler 37a05372fa fix: disable relay if built-in DERP is disabled (#12654)
Fixes https://github.com/coder/coder/issues/12493
2024-03-21 16:53:41 -05:00
Kyle Carberry d3c9aaf57b chore: update hero image for refreshed dashboard (#12712)
* chore: update hero image for refreshed dashboard

I used html.to.figma for this. It's in the following Figma:
https://www.figma.com/file/m4zkIU7e64BNyRlUShuluq/README-Screenshot?type=design&node-id=13-1993&mode=design&t=o2AhL8y7u8uFMVPI-0

I couldn't take screenshots directly, because I'm on Linux and the macOS
browser with the icons look much better.

* Try stacked images

* Try stacked inverted
2024-03-21 16:28:02 -04:00
Cian Johnston 28730ca3d8 fix(support): sanitize manifest (#12711) 2024-03-21 19:55:34 +00:00
Cian Johnston f2a9e515df feat(cli/support): confirm before creating bundle (#12684)
Forces user to confirm before creating a support bundle.
Also adds contextual information to the bundle under cli_logs.txt.
2024-03-21 17:06:28 +00:00
Cian Johnston 8ea5fb7115 chore(coderd/workspaceapps/apptest): fix test flake due to concurrent usage of same deployment (#12700)
- Updates existing tests under workspaceapps/apptest to not reuse existing appDetails as assertWorkspaceLastUsed(Not)?Updated calls FlushStats() which was causing racy test behaviour and incorrect test assertions.
- Expands scope of assertWorkspaceLastUsedAtUpdated and its counterpart to ProxySubdomain tests.
2024-03-21 15:38:38 +00:00
Cian Johnston 5454f4997b chore(ci): clean up databases after test finishes in CI (#12702) 2024-03-21 14:53:16 +00:00
Mathias Fredriksson 9c84fb7fb1 fix(coderd/agentapi): always write agent stats when provided (#12699) 2024-03-21 16:47:06 +02:00
Steven Masley bd6ad88077 chore: nolint always return error function (#12701) 2024-03-21 09:35:10 -05:00
Steven Masley b4492fffba chore: support multiple key:value search query params (#12690)
This more closely aligns with GitHub's label search style. Actual search params need to be converted to allow this format, by default they will throw an error if they do not support listing.
2024-03-21 08:37:19 -05:00
Bruno Quaresma 8499eacf67 chore(site): add tests for deprecate template flow (#12685)
Closes #12505
2024-03-21 10:37:08 -03:00
Steven Masley 131d0bd2ba chore: fix linting issue in main(#12697) 2024-03-20 20:15:01 -05:00
dependabot[bot] f93491ffe8 chore: bump github.com/docker/docker from 24.0.7+incompatible to 24.0.9+incompatible (#12692)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-21 02:14:30 +03:00
Dean Sheather 2b773f9034 fix: allow proxy version mismatch (with warning) (#12433) 2024-03-20 18:24:18 +00:00
Garrett Delfosse 4d9fe05f5a feat: add awsiamrds db auth driver (#12566) 2024-03-20 13:14:43 -04:00
Ammar Bandukwala 0d86dca852 fix(codersdk): abort in-progress writes/reads when closing websocket (#12650)
Fixes #9203

Related #12065

Also, adds some basic tracing infrastructure that we can build upon for more improvements.
2024-03-20 11:53:32 -05:00
Cian Johnston 92aa1eba97 fix(cli): port-forward: update workspace last_used_at (#12659)
This PR updates the coder port-forward command to periodically inform coderd that the workspace is being used:

- Adds workspaceusage.Tracker which periodically batch-updates workspace LastUsedAt
- Adds coderd endpoint to signal workspace usage
- Updates coder port-forward to periodically hit this endpoint
- Modifies BatchUpdateWorkspacesLastUsedAt to avoid overwriting with stale data

Co-authored-by: Danny Kopping <danny@coder.com>
2024-03-20 16:44:12 +00:00
Steven Masley d789a60d47 chore: remove max_ttl from templates (#12644)
* chore: remove max_ttl from templates

Completely removing max_ttl as a feature on template scheduling. Must use other template scheduling features to achieve autostop.
2024-03-20 10:37:57 -05:00
Bruno Quaresma d82e20152b feat(site): make listening ports scrollable (#12660) 2024-03-20 09:34:30 -03:00
Kayla Washburn-Love 9028717c9b fix: disable auto-create if external auth requirements aren't met (#12538) 2024-03-19 16:42:40 -06:00
Muhammad Atif Ali ef26ad96a9 chore(dogfood): bump modules versions 2024-03-19 23:54:16 +03:00
Kyle Carberry 4ae1f40eee chore: add docs for adding e2e tests (#12677) 2024-03-19 18:25:05 +00:00
Ben Potter c92ceffac9 chore: fix changelog typos (#12663) 2024-03-19 16:04:27 +00:00
Steven Masley 00283d1f8b chore: helm golden file test log output on failure (#12661)
Debugging this is hard without the actual helm error
2024-03-19 16:01:54 +00:00
Bruno Quaresma 23e3e4ce58 chore(site): upgrade msw to 2.0 (#12597)
Closes https://github.com/coder/coder/issues/11426
2024-03-19 09:30:20 -03:00
Danny Kopping 9cfd5baa91 feat(coderd): export metric indicating each experiment's status (#12657) 2024-03-19 14:11:27 +02:00
Ben Potter 1a9f7e7b00 hotfix(docs): we do not offer phone support yet (#12658) 2024-03-19 10:22:32 +00:00
Danny Kopping ab95ae827d feat(coderd): add enabled experiments to telemetry (#12656) 2024-03-19 11:05:29 +02:00
Steven Masley f0f9569d51 chore: enforce that provisioners can only acquire jobs in their own organization (#12600)
* chore: add org ID as optional param to AcquireJob
* chore: plumb through organization id to provisioner daemons
* add org id to provisioner domain key
* enforce org id argument
* dbgen provisioner jobs defaults to default org
2024-03-18 12:48:13 -05:00
Marcin Tojek 0e8ebb9b22 fix: fix flaky TestWorkspaceProxy_Server_PrometheusEnabled (#12645) 2024-03-18 16:43:41 +01:00
dependabot[bot] 2cd5fbc712 chore: bump github.com/ammario/tlru from 0.3.0 to 0.4.0 (#12635)
Bumps [github.com/ammario/tlru](https://github.com/ammario/tlru) from 0.3.0 to 0.4.0.
- [Commits](https://github.com/ammario/tlru/compare/v0.3.0...v0.4.0)

---
updated-dependencies:
- dependency-name: github.com/ammario/tlru
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 11:25:52 -04:00
dependabot[bot] 124da2e51c chore: bump github.com/u-root/u-root from 0.13.0 to 0.14.0 (#12636)
Bumps [github.com/u-root/u-root](https://github.com/u-root/u-root) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/u-root/u-root/releases)
- [Changelog](https://github.com/u-root/u-root/blob/main/RELEASES)
- [Commits](https://github.com/u-root/u-root/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: github.com/u-root/u-root
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 11:25:35 -04:00
Marcin Tojek cae769eac0 fix: implicit schema in dump (#12646) 2024-03-18 16:25:08 +01:00
dependabot[bot] 543a8ccb31 chore: bump github.com/coder/serpent (#12637)
Bumps [github.com/coder/serpent](https://github.com/coder/serpent) from 0.4.1-0.20240315163851-a0148c87ea3f to 0.5.0.
- [Commits](https://github.com/coder/serpent/commits/v0.5.0)

---
updated-dependencies:
- dependency-name: github.com/coder/serpent
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 09:39:28 -05:00
Muhammad Atif Ali f0c5e8e960 chore: reduce slack spam for manually run docs link checker workflows (#12630) 2024-03-18 17:23:54 +03:00
Marcin Tojek 15845d1a65 chore: use sqlc-vet to verify schema (#12642) 2024-03-18 15:23:25 +01:00
Ammar Bandukwala e5cc17af92 chore(cli): hide --organization (#12626)
It's potentially confusing to users since we aren't fleshing out organizations right now.
2024-03-18 09:09:26 -05:00
Dean Sheather cf50461ab4 fix: prevent single replica proxies from staying unhealthy (#12641)
In the peer healthcheck code, when an error pinging peers is detected we
write a "replicaErr" string with the error reason. However, if there are
no peer replicas to ping we returned early without setting the string to
empty. This would cause replicas that had peers (which were failing) and
then the peers left to permanently show an error until a new peer
appeared.

Also demotes DERP replica checking to a "warning" rather than an "error"
which should prevent the primary from removing the proxy from the region
map if DERP meshing is non-functional. This can happen without causing
problems if the peer is shutting down so we don't want to disrupt
everything if there isn't an issue.
2024-03-18 23:45:25 +10:00
Kyle Carberry 8a2f38a746 chore: simplify readme (#12639)
* chore: simplify readme

Closes #12628.

* Add link to module registry
2024-03-18 09:40:47 -04:00
Bruno Quaresma c84d96b747 fix(site): display not found page when pagination page is invalid (#12611) 2024-03-18 10:35:59 -03:00
dependabot[bot] b121f407f5 chore: bump golang.org/x/tools from 0.18.0 to 0.19.0 (#12632)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 14:44:16 +03:00
dependabot[bot] eb20d8cf18 chore: bump golang.org/x/mod from 0.15.0 to 0.16.0 (#12633)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 14:44:05 +03:00
dependabot[bot] fffa3dc422 chore: bump github.com/gohugoio/hugo from 0.123.3 to 0.124.0 (#12634)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 14:43:58 +03:00
Danny Kopping 93933d7905 feat(cli): show queue position during workspace builds (#12606) 2024-03-18 12:05:05 +02:00
dependabot[bot] c7597fdf02 chore: bump go.nhat.io/otelsql from 0.12.0 to 0.13.0 (#12521)
Bumps [go.nhat.io/otelsql](https://github.com/nhatthm/otelsql) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/nhatthm/otelsql/releases)
- [Commits](https://github.com/nhatthm/otelsql/compare/v0.12.0...v0.13.0)

---
updated-dependencies:
- dependency-name: go.nhat.io/otelsql
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 11:03:45 +03:00
dependabot[bot] 77cc170f04 chore: bump google.golang.org/api from 0.152.0 to 0.170.0 (#12607)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.152.0 to 0.170.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.152.0...v0.170.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 10:45:52 +03:00
Danny Kopping 53b58ed74b fix: correct troubleshooting links (#12608) 2024-03-18 08:52:20 +02:00
elasticspoon 5011edc292 fix(cli): show error/hide help for unsupported subcommands (#10760) (#12624)
Fixes #10760 

The coder CLI quietly accepts any subcommand arguments and silently swallows them.

Currently:
```sh
❯ coder | head -n5
coder v2.3.3+e491217

USAGE:
  coder [global-flags] <subcommand>
```

```sh
❯ coder idontexist | head -n5
coder v2.3.3+e491217

USAGE:
  coder [global-flags] <subcommand>
```

Now help output will not be show when there is an unknown subcommand error. Instead users will be given the command for the help output.

```sh
❯ coder idontexist
Encountered an error running "coder", see "coder --help" for more information 
error: unrecognized subcommand "idontexist"
```
```sh
❯ coder iexistbut idontexist
Encountered an error running "coder iexistbut", see "coder iexistbut --help" for more information 
error: unrecognized subcommand "idontexist"
```

Also this stuff: `Encountered an error running "coder iexistbut"... ` gets written to `os.Stdout` in `prettyErrorFormatter{w: os.Stderr, verbose: r.verbose}`, not sure how to test that output.
2024-03-17 22:17:43 -05:00
Steven Masley c189cc93e4 chore: bump gopkg.in/DataDog/dd-trace-go.v1 from 1.57.0 to 1.61.0 (#12610)
* chore: bump gopkg.in/DataDog/dd-trace-go.v1 from 1.57.0 to 1.61.0

* Fix tracer implementation
* Use alias vs 2 structs
2024-03-17 22:08:32 -05:00
Ammar Bandukwala b4c0fa80d8 chore(cli): rename Cmd to Command (#12616)
I think Command is cleaner and my original decision to use "Cmd"
a mistake.

Plus this creates better parity with cobra.
2024-03-17 09:45:26 -05:00
Muhammad Atif Ali 2a77580ba6 chore: fix false positives for docs links checker (#12623) 2024-03-17 16:44:45 +03:00
dependabot[bot] aa3ab209f3 ci: bump the github-actions group with 3 updates (#12622)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-17 13:27:15 +00:00
Gary Wang 1a5c5d0d57 fix: correct typo error about minTerraformVersion (#12621) 2024-03-17 13:18:46 +00:00
dependabot[bot] 4bdb019001 chore: bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#12614)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-17 16:12:51 +03:00
Gábor 9c69672382 fix(migration): removed hardcoded public (#12620) 2024-03-16 10:11:14 -04:00
Cian Johnston 9ff0bafcee fix(support): also sanitize agent environment (#12615) 2024-03-15 20:19:35 +00:00
Bruno Quaresma 6f0ba5bfe7 chore(site): add AgentLogs storybook (#12601) 2024-03-15 14:57:35 -03:00
Cian Johnston 25b605f764 fix(examples/lima/coder.yaml): update base image, remove usage of deprecated LIMA_CIDATA (#12613)
* fix(examples/lima/coder.yaml): update base image, remove usage of deprecated LIMA_CIDATA, name template consistently

* make fmt
2024-03-15 17:22:44 +00:00
Ammar Bandukwala 496232446d chore(cli): replace clibase with external coder/serpent (#12252) 2024-03-15 11:24:38 -05:00
Marcin Tojek bed2545636 docs: describe reference architectures (#12609) 2024-03-15 17:01:45 +01:00
Cian Johnston b0c4e7504c feat(support): add client magicsock and agent prometheus metrics to support bundle (#12604)
* feat(codersdk): add ability to fetch prometheus metrics directly from agent
* feat(support): add client magicsock and agent prometheus metrics to support bundle
* refactor(support): simplify AgentInfo control flow

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2024-03-15 15:33:49 +00:00
Eric Paulsen 4d9e6c0134 add updated architecture diagrams (#12584) 2024-03-15 11:09:03 -04:00
Cian Johnston 2fc9f097ed chore: apply linter auto-fixes (#12605) 2024-03-15 14:39:25 +00:00
Cian Johnston 18c1e02bf0 dogfood: replace siegfried with greenhill (#12599) 2024-03-15 23:33:43 +10:00
dependabot[bot] e1685b96e4 chore: bump golang.org/x/tools from 0.18.0 to 0.19.0 (#12519)
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.18.0 to 0.19.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 13:18:35 +00:00
Kyle Carberry 895df54051 fix: separate signals for passive, active, and forced shutdown (#12358)
* fix: separate signals for passive, active, and forced shutdown

`SIGTERM`: Passive shutdown stopping provisioner daemons from accepting new
jobs but waiting for existing jobs to successfully complete.

`SIGINT` (old existing behavior): Notify provisioner daemons to cancel in-flight jobs, wait 5s for jobs to be exited, then force quit.

`SIGKILL`: Untouched from before, will force-quit.

* Revert dramatic signal changes

* Rename

* Fix shutdown behavior for provisioner daemons

* Add test for graceful shutdown
2024-03-15 13:16:36 +00:00
dependabot[bot] 2c947c1921 chore: bump golang.org/x/mod from 0.15.0 to 0.16.0 (#12520)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.15.0 to 0.16.0.
- [Commits](https://github.com/golang/mod/compare/v0.15.0...v0.16.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 13:09:50 +00:00
dependabot[bot] 78f26bf24a chore: bump github.com/u-root/u-root from 0.13.0 to 0.14.0 (#12397)
Bumps [github.com/u-root/u-root](https://github.com/u-root/u-root) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/u-root/u-root/releases)
- [Changelog](https://github.com/u-root/u-root/blob/main/RELEASES)
- [Commits](https://github.com/u-root/u-root/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: github.com/u-root/u-root
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 08:55:10 -04:00
dependabot[bot] cf7f95b418 chore: bump golang.org/x/oauth2 from 0.17.0 to 0.18.0 (#12528)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.17.0 to 0.18.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 08:54:32 -04:00
dependabot[bot] 351706b896 chore: bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#12524)
Bumps google.golang.org/protobuf from 1.32.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 08:54:26 -04:00
Cian Johnston 2abc1cd2b7 feat(support): fetch agent network info over tailnet (#12577)
Adds agent-related information to support bundle command.
2024-03-15 11:01:39 +00:00
Cian Johnston 653ddccd8e fix(agent): remove unused token debug handler (#12602) 2024-03-15 09:43:36 +00:00
dependabot[bot] 8d7819f6d6 chore: bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#12404)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-14 19:37:32 -05:00
Steven Masley 29c8cf20e0 fix: handle CLI default organization when none exists in <v2.9.0 coderd (#12594)
* chore: 'coder org set' help message was incorrect
* fix: handler coder cli against older versions of Coder
2024-03-14 15:11:29 -05:00
Bruno Quaresma f78b5c1cfe chore(site): refactor logs and add stories (#12553) 2024-03-14 14:49:37 -03:00
Garrett Delfosse 0723dd3abf fix: ensure agent token is from latest build in middleware (#12443) 2024-03-14 12:27:32 -04:00
Cian Johnston 63696d762f feat(codersdk): add debug handlers for logs, manifest, and token to agent (#12593)
* feat(codersdk): add debug handlers for logs, manifest, and token to agent

* add more logging

* use io.LimitReader instead of seeking
2024-03-14 15:36:12 +00:00
Cian Johnston 135381bb4e chore(cli): skip support bundle test on windows (#12596) 2024-03-14 15:25:09 +00:00
Mathias Fredriksson 5dd436c19b feat(examples): add linting to all examples (#12595)
Fixes #12588
2024-03-14 16:49:44 +02:00
Steven Masley 410a7d54ee chore: external auth flow opens new window, it does not need an href (#12586) 2024-03-14 09:17:50 -05:00
Steven Masley 4cba83b30f test: apptest was accidently choosing ports in use (#12580)
Apptest requires a port without a listening server to test failure
cases. This port was chosen and had a chance of actually being
provisioned. To prevent this accident, a port <1k is chosen,
since those will never be allocated.
2024-03-14 08:54:12 -05:00
Danny Kopping 14130deb07 fix: clean template destination path for pull (#12559) 2024-03-14 12:41:23 +00:00
Cian Johnston 395bf54f4f chore(examples/templates/incus): fix metadata harder (#12589) 2024-03-14 11:43:34 +00:00
Muhammad Atif Ali 04b711f187 chore(examples/templates/incus): fix metadata (#12587) 2024-03-14 11:14:37 +00:00
Cian Johnston 3b406878e0 feat(agent): expose HTTP debug server over tailnet API (#12582) 2024-03-14 10:02:01 +00:00
Asher 0d16df9df9 fix: importing api into vscode-coder (#12570) 2024-03-13 10:05:44 -08:00
Kayla Washburn-Love efba477b36 fix: hide actions and notifications from deleted workspaces (#12563) 2024-03-13 11:43:48 -06:00
Bruno Quaresma 489b0ec497 chore(site): refactor useAuth and related hooks (#12567)
Close https://github.com/coder/coder/issues/12487
2024-03-13 13:24:18 -03:00
Spike Curtis fe6def31eb feat: upgrade tailscale fork to set TCP options for perf (#12574)
Upgrades our tailscale fork to include some TCP options tweaks for performance. c.f. https://github.com/coder/tailscale/pull/46
2024-03-13 16:02:02 +04:00
Michael Brewer 903f8b21c4 feat(site): add cpp icon (#12572)
* feat(site): add cpp icon

Add the C++ icon

Source is WikiCommons: https://en.m.wikipedia.org/wiki/File:ISO_C%2B%2B_Logo.svg

* fix: correct order
2024-03-13 07:49:03 -04:00
Danny Kopping da54c8a51f fix: fix data race in TestLabelsAggregation tests (#12578) 2024-03-13 13:47:22 +02:00
Danny Kopping 7a7105ad66 feat: make agent stats' cardinality configurable (#12535) 2024-03-13 12:03:36 +02:00
Bruno Quaresma e45d511f28 chore(site): add missing stories for templates (#12537) 2024-03-12 17:54:37 -03:00
Kayla Washburn-Love 301c60d824 chore(dogfood): add fish and helix ppa packages to dogfood (#12568) 2024-03-12 13:58:40 -06:00
Cian Johnston 096d472de9 chore(dogfood): update remaining focal PPAs to jammy (#12565) 2024-03-12 19:24:58 +00:00
Cian Johnston 901668ad4b feat(dogfood): add git from git-core ppa (#12564) 2024-03-12 17:28:07 +00:00
Bruno Quaresma 8489b4dfb1 chore(site): add WorkspaceBuildData stories (#12550) 2024-03-12 14:18:31 -03:00
Bruno Quaresma e947e0e829 chore(site): refactor dormant badge and add stories (#12555) 2024-03-12 14:17:55 -03:00
Ben Potter 321546474b docs: add v2.9.0 changelog (#12562)
* docs: add v2.9.0 changelog

* added sharable ports screenshot

* moved autostop visibility improvements from backend to dashboard, added screenshot

* move experiments to bottom

* added activity bump screenshot

---------

Co-authored-by: Stephen Kirby <me@skirby.dev>
2024-03-12 12:12:35 -05:00
Cian Johnston 47cb584052 fix(support): sanitize agent env (#12554) 2024-03-12 15:23:11 +00:00
Steven Masley 597694fbdd chore: bump migration file (#12556) 2024-03-12 14:55:45 +00:00
Steven Masley e11d3ca0ee chore: move default everyone group to a migration (#12435) 2024-03-12 09:27:36 -05:00
Bruno Quaresma f3083226ab chore: add package manager (#12551)
Every time I run `pnpm` in the project it adds the package manager attribute on package.json so I just decided to push it since it does not look like an issue and we can make sure everyone is running the same pnpm version.
2024-03-12 10:44:23 -03:00
Cian Johnston 7b081c873e fix(site): warn when user leaves template editor with un-built changes (#12548) 2024-03-12 13:08:54 +00:00
Danny Kopping 90d00190ea chore: remove pr_number param from deploy-pr.sh (#12549)
Field was removed in https://github.com/coder/coder/pull/11259

Signed-off-by: Danny Kopping <danny@coder.com>
2024-03-12 12:01:31 +00:00
Cian Johnston edc465c449 fix(site): TemplateVersionEditor: allow triggering builds on non-dirtied template version (#12547) 2024-03-12 11:35:16 +00:00
Spike Curtis 51707446d0 fix: stop holding Pubsub mutex while calling pq.Listener (#12518)
fixes #11950

https://github.com/coder/coder/issues/11950#issuecomment-1987756088 explains the bug

We were also calling into `Unlisten()` and `Close()` while holding the mutex.  I don't believe that `Close()` depends on the notification loop being unblocked, but it's hard to be sure, and the safest thing to do is assume it could block.

So, I added a unit test that fakes out `pq.Listener` and sends a bunch of notifies every time we call into it to hopefully prevent regression where we hold the mutex while calling into these functions.

It also removes the use of a `context.Context` to stop the PubSub -- it must be explicitly `Closed()`.  This simplifies a bunch of the logic, and is how we use the pubsub anyway.
2024-03-12 09:44:12 +04:00
dependabot[bot] 6f00ccfa64 chore: bump storj.io/drpc from 0.0.33-0.20230420154621-9716137f6037 to 0.0.33 (#12526)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-12 03:43:01 +00:00
dependabot[bot] da146e9655 chore: bump golang.org/x/crypto from 0.20.0 to 0.21.0 (#12523)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-12 03:41:54 +00:00
dependabot[bot] 242e4c4c85 chore: bump golang.org/x/term from 0.17.0 to 0.18.0 (#12525)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-12 03:40:00 +00:00
Michael Brewer f6ed81bc3b feat(site): add microsoft teams icon (#12513)
* feat(site): add microsoft teams icon

Attribution below:
Vectors and icons by <a href="https://github.com/garudatechnologydevelopers/sketch-icons?ref=svgrepo.com" target="_blank">Garuda Technology</a> in MIT License via <a href="https://www.svgrepo.com/" target="_blank">SVG Repo</a>


closes #12496

* fix(site): correct order of icons
2024-03-12 03:18:10 +00:00
Cian Johnston d704ff4570 chore(codersdk): explain format of codersdk.UpdateWorkspaceAutostartRequest.Schedule (#12539) 2024-03-11 22:50:38 +00:00
Kayla Washburn-Love a546cb8b32 chore: add stories to Search (#12457) 2024-03-11 12:16:31 -06:00
Bruno Quaresma 83af8674e8 chore(site): add CreateTokenPage story (#12472) 2024-03-11 14:10:19 -03:00
Steven Masley e3051dff0c chore: add workspace id filter on api (#12483)
* chore: add workspace id filter on api
2024-03-11 11:37:15 -05:00
Cian Johnston 8f40ee3465 Revert "feat: make agent stats' cardinality configurable (#12468)" (#12533)
This reverts commit 21d1873d97.
2024-03-11 14:33:36 +00:00
elasticspoon 773862a9f5 feat(cli): make url optional for login command (#10925) (#12466)
Allow `coder login` to log into existing deployment if available.

Update help and error messages to indicate that `coder login` is
available as a command.

Fixes #10925
Fixes #9551
2024-03-11 16:14:19 +02:00
Cian Johnston bed61f7d2a fix(coderd): correctly handle tar dir entries with missing path separator (#12479)
* coderd: add test to reproduce trailing directory issue
* coderd: add trailing path separator to dir entries when converting to zip
* provisionersdk: add trailing path separator to directory entries
2024-03-11 14:06:41 +00:00
Danny Kopping 21d1873d97 feat: make agent stats' cardinality configurable (#12468)
Closes #12221
2024-03-11 16:04:08 +02:00
Cian Johnston 0647ec1960 fix(coderd): prevent nil err deref (#12475) 2024-03-11 14:03:58 +00:00
Garrett Delfosse dc69341583 fix: make public menu item selectable (#12484) 2024-03-11 10:00:40 -04:00
Alessandro Varesi 5e9bf31229 fix: devcontainer-docker bad default directory (#12453) 2024-03-11 16:56:41 +03:00
Michael Brewer cef632b1fb feat(site): add dotnet icon (#12512) 2024-03-11 16:54:34 +03:00
Bruno Quaresma cd64e981b4 chore(site): add stories to 404 page (#12470)
Related to https://github.com/coder/coder/issues/12263
2024-03-11 10:36:06 -03:00
Cian Johnston b1ecc53033 chore(coderd): improve tests for tar<->zip conversion (#12477)
* improve tests for tar<->zip conversion
* set mode and modtime correctly when converting from zip to tar (#12476)
2024-03-11 13:29:57 +00:00
Bruno Quaresma 0220c97ef9 chore(site): add TableToolbar stories (#12473)
Related to https://github.com/coder/coder/issues/12263
2024-03-11 10:21:47 -03:00
Bruno Quaresma b8dd6b3aa2 chore(site): add Form storybook (#12469)
Related to #12260
2024-03-11 10:21:18 -03:00
Cian Johnston 1f276a22b3 chore(dogfood): update keys (#12515) 2024-03-11 13:07:48 +00:00
Mathias Fredriksson bae0a747ed test(coderd): skip flaky dau test (#12517)
* test(coderd): skip flaky dau test

* chore(coderd/database/dbpurge): fix failing test (#12530)

---------

Co-authored-by: Cian Johnston <cian@coder.com>
2024-03-11 12:54:38 +00:00
Michael Brewer 5296611a3f feat(site): add confluence icon (#12500)
Attribution for this icon below, if needed

Vectors and icons by <a href="https://github.com/vscode-icons/vscode-icons?ref=svgrepo.com" target="_blank">Vscode Icons</a> in MIT License via <a href="https://www.svgrepo.com/" target="_blank">SVG Repo</a>
2024-03-11 11:34:02 +03:00
dependabot[bot] 2b4560cc4b chore: bump github.com/fergusstrange/embedded-postgres (#12400)
Bumps [github.com/fergusstrange/embedded-postgres](https://github.com/fergusstrange/embedded-postgres) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/fergusstrange/embedded-postgres/releases)
- [Commits](https://github.com/fergusstrange/embedded-postgres/compare/v1.25.0...v1.26.0)

---
updated-dependencies:
- dependency-name: github.com/fergusstrange/embedded-postgres
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-11 00:16:16 +03:00
dependabot[bot] 6588cee38a chore: bump github.com/go-jose/go-jose/v3 from 3.0.1 to 3.0.3 (#12460)
Bumps [github.com/go-jose/go-jose/v3](https://github.com/go-jose/go-jose) from 3.0.1 to 3.0.3.
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Changelog](https://github.com/go-jose/go-jose/blob/v3.0.3/CHANGELOG.md)
- [Commits](https://github.com/go-jose/go-jose/compare/v3.0.1...v3.0.3)

---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-11 00:15:52 +03:00
Michael Smith 4d42c07c72 chore(site): update and refactor all custom hook tests that rely on React Router (#12219)
* chore: rename useTab to useSearchParamsKey and add test

* chore: mark old renderHookWithAuth as deprecated (temp)

* fix: update imports for useResourcesNav

* refactor: change API for useSearchParamsKey

* chore: let user pass in their own URLSearchParams value

* refactor: clean up comments for clarity

* fix: update import

* wip: commit progress on useWorkspaceDuplication revamp

* chore: migrate duplication test to new helper

* refactor: update code for clarity

* refactor: reorder test cases for clarity

* refactor: split off hook helper into separate file

* refactor: remove reliance on internal React Router state property

* refactor: move variables around for more clarity

* refactor: more updates for clarity

* refactor: reorganize test cases for clarity

* refactor: clean up test cases for useWorkspaceDupe

* refactor: clean up test cases for useWorkspaceDupe
2024-03-08 18:31:01 -05:00
Kayla Washburn-Love cf4f56dc2f chore: add stories for MoreMenu (#12464) 2024-03-08 12:01:48 -07:00
Kayla Washburn-Love 8d8220bb07 chore: add stories for Loader (#12445) 2024-03-08 11:35:14 -07:00
Ben Potter 1e17782ff6 docs: simplify install docs (#11946)
* docs: simplify install docs

* changes from feedback

* fmt

* fixups from feedback
2024-03-08 15:15:59 +00:00
Danny Kopping 7a92154e67 Install pnpm before calling pnpm exec in make stages (#12471)
Signed-off-by: Danny Kopping <danny@coder.com>
2024-03-08 14:33:28 +02:00
Bruno Quaresma 5b2acbc5b7 chore(site): add FileUpload stories (#12456)
Related to https://github.com/coder/coder/issues/12260
2024-03-08 09:08:44 -03:00
Bruno Quaresma 18d1c17db1 chore(site): add storybook for BuildAvatar and BuildIcon (#12455)
Related to https://github.com/coder/coder/issues/12260
2024-03-08 08:06:56 -03:00
Bruno Quaresma 060033e4ef fix(site): fix terminal size when displaying alerts (#12444)
Before - The terminal size does not fit the available space so the bottom is hidden.

https://github.com/coder/coder/assets/3165839/d08470b9-9fc6-476c-a551-8a3e13fc25bf

After - The terminal adjusts when there are alert changes.

https://github.com/coder/coder/assets/3165839/8cc32bfb-056f-47cb-97f2-3bb18c5fe906

Unfortunately, I don't think there is a sane way to automate tests for this but open to suggestions.

Close https://github.com/coder/coder/issues/7914
2024-03-08 07:38:40 -03:00
Dean Sheather d2a5b31b2b feat: add derp mesh health checking in workspace proxies (#12222) 2024-03-08 16:31:40 +10:00
Colin Adler 6b0b87eb27 fix: add --block-direct-connections to wsproxies (#12182) 2024-03-07 23:45:59 -06:00
Colin Adler 66154f937e fix(coderd): pass block endpoints into servertailnet (#12149) 2024-03-08 05:29:54 +00:00
Garrett Delfosse d2a74cf547 fix: display tooltip when selection is disabled (#12439) 2024-03-07 10:43:25 -05:00
Dean Sheather 586586e9dd fix: do not set max deadline for workspaces on template update (#12446)
* fix: do not set max deadline for workspaces on template update

When templates are updated and schedule data is changed, we update all
running workspaces to have up-to-date scheduling information that sticks
to the new policy.

When updating the max_deadline for existing running workspaces, if the
max_deadline was before now()+2h we would set the max_deadline to
now()+2h.

Builds that don't/shouldn't have a max_deadline have it set to 0, which
is always before now()+2h, and thus would always have the max_deadline
updated.

* test: add unit test to excercise template schedule bug
---------

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
2024-03-07 09:42:50 -06:00
Cian Johnston 17caf58b5e feat(support): add template info to support bundle (#12451)
Adds workspace build parameters, template, template version, and zipped template source to support bundle.
2024-03-07 14:43:46 +00:00
Bruno Quaresma db02c72ac6 chore(site): add storybook for terminal page (#12441) 2024-03-07 14:17:38 +00:00
Spike Curtis b96f6b48a4 fix: ensure ssh cleanup happens on cmd error
I noticed in my logs that sometimes `coder ssh` doesn't gracefully disconnect from the coordinator.

The cause is the `closerStack` construct we use in that function.  It has two paths to start closing things down:

1. explicit `close()` which we do in `defer`
2. context cancellation, which happens if the cli function returns an error

sometimes the ssh remote command returns an error, and this triggers context cancellation of the `closerStack`.  That is fine in and of itself, but we still want the explicit `close()` to wait until everything is closed before returning, since that's where we do cleanup, including the graceful disconnect.  Prior to this fix the `close()` just immediately exits if another goroutine is closing the stack.  Here we add a wait until everything is done.
2024-03-07 17:26:49 +04:00
Cian Johnston c8aa99a5b8 feat(coderd/database/dbfake): allow specifying fileID in TemplateVersionBuilder (#12450) 2024-03-07 12:36:11 +00:00
dependabot[bot] e4326947c4 chore: bump github.com/go-chi/httprate from 0.8.0 to 0.9.0 (#12401)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-07 16:57:21 +05:00
dependabot[bot] 14b1400968 chore: bump github.com/go-playground/validator/v10 from 10.18.0 to 10.19.0 (#12396)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-07 16:57:12 +05:00
dependabot[bot] de35755bd2 chore: bump github.com/hashicorp/hcl/v2 from 2.17.0 to 2.20.0 (#12398)
Bumps [github.com/hashicorp/hcl/v2](https://github.com/hashicorp/hcl) from 2.17.0 to 2.20.0.
- [Release notes](https://github.com/hashicorp/hcl/releases)
- [Changelog](https://github.com/hashicorp/hcl/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/hcl/compare/v2.17.0...v2.20.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/hcl/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-07 10:46:51 +00:00
Dean Sheather bd752a6d8b chore: embed static files in debug builds (#12449) 2024-03-07 09:23:28 +00:00
Cian Johnston 3e6e1e6f10 ci: add jnb wsproxy + update dogfood template (#12440)
Co-authored-by: Dean Sheather <dean@deansheather.com>
2024-03-06 18:54:26 +00:00
Dean Sheather 662be56d72 chore: rename migrations to fix main (#12442) 2024-03-06 18:28:53 +00:00
Steven Masley b5f866c1cb chore: add organization_id column to provisioner daemons (#12356)
* chore: add organization_id column to provisioner daemons
* Update upsert to include organization id on set
2024-03-06 12:04:50 -06:00
Dean Sheather 46a2ff1061 feat: allow setting port share protocol (#12383)
Co-authored-by: Garrett Delfosse <garrett@coder.com>
2024-03-06 09:23:57 -05:00
Steven Masley 23ff807a27 chore: remove autocreate orgs on CreateUser (#12434)
New users must be explictly given an organization to join.
Organizations should not be auto created as a side effect of
creating a new user.
2024-03-06 07:29:28 -06:00
Dean Sheather 842799847a chore: fix trivy scanning (#12421) 2024-03-05 19:04:16 -06:00
Michael Smith a92853c72d fix: ensure auto-workspace creation waits until all parameters are ready (#12419)
* fix: ensure auto-workspace creation waits until all parameters are ready

* refactor: move creation blocking logic to main callback

* fix: let creation start if experimental feature is off
2024-03-05 18:42:50 -05:00
Kayla Washburn-Love 0fe109d517 chore: sort imports in our typescript code (#12417) 2024-03-05 16:31:22 -07:00
Steven Masley 17c486c5e6 chore: ensure default org always exists (#12412)
* chore: ensure default org always exists

First user just joins the org created by the migration
2024-03-05 14:06:35 -06:00
Bruno Quaresma bc30c9c013 feat(site): warn user if they leave the editor without publishing (#12406) 2024-03-05 16:55:23 -03:00
Garrett Delfosse 61bd341a36 chore: change max share level on existing port shares (#12411) 2024-03-05 13:47:01 -05:00
Cian Johnston 5106d9fc47 feat(support): fetch data concurrently (#12385)
Modifies pkg support to fetch data concurrently
2024-03-05 17:41:42 +00:00
Bruno Quaresma fb88fa8603 feat(site): display error messages on ws and access url health pages (#12430)
Close https://github.com/coder/coder/issues/12408
2024-03-05 13:27:57 -03:00
Cian Johnston 4343998c37 chore(coderd): add tests for big oidc tokens (#12424)
- Adds two test cases for a 64k+ ID token and a 64k+ userinfo payload.
- Reformats the entire test cases array as instructed by CI
2024-03-05 14:46:00 +00:00
Marcin Tojek b1f9a6dc31 fix: use timestamptz instead of timestamp (#12425)
* fix: use timestampz instead of timestamp

* fix: timestamptz
2024-03-05 14:16:29 +00:00
Marcin Tojek 3e99c0373f fix: improve pagination parser (#12422) 2024-03-05 14:05:15 +00:00
Cian Johnston 61db293b33 feat(scripts/develop.sh): add --debug flag to develop.sh (#12423)
Adds a `--debug` flag to `scripts/develop.sh` that will start coder under `dlv debug` instead.
You can then use e.g. the following launch snippet to connect dlv:
```
    {
      "name": "Delve Remote",
      "type": "go",
      "request": "attach",
      "mode": "remote",
      "port": 12345,
    }
```

You can also run invididual CLI commands under dlv e.g.

```
debug=1 scripts/coder-dev.sh list
```

Also sets CGO_ENABLED=0 in develop.sh by default.
2024-03-05 13:29:08 +00:00
dependabot[bot] 8585863d0e chore: bump golang.org/x/crypto from 0.19.0 to 0.20.0 (#12403)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.19.0 to 0.20.0.
- [Commits](https://github.com/golang/crypto/compare/v0.19.0...v0.20.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-05 15:49:17 +05:00
Marcin Tojek e4fa212164 fix: always return count of workspaces (#12407) 2024-03-05 09:24:43 +01:00
Dean Sheather 0016b0200b chore: add test for workspace proxy derp meshing (#12220)
- Reworks the proxy registration loop into a struct (so I can add a `RegisterNow` method)
- Changes the proxy registration loop interval to 15s (previously 30s)
- Adds test which tests bidirectional DERP meshing on all possible paths between 6 workspace proxy replicas

Related to https://github.com/coder/customers/issues/438
2024-03-04 23:40:15 -08:00
Steven Masley 5c6974e55f feat: implement provisioner auth middleware and proper org params (#12330)
* feat: provisioner auth in mw to allow ExtractOrg

Step to enable org scoped provisioner daemons

* chore: handle default org handling for provisioner daemons
2024-03-04 15:15:41 -06:00
Colin Adler 926fd7ffa6 chore: add v2.6.1, v2.7.3, v2.8.4 release notes (#12415) 2024-03-04 12:23:01 -06:00
Alex 320c2eac6f Entra External Auth for ADO (#12201) 2024-03-04 12:12:46 -06:00
Colin Adler 4439a920e4 Merge pull request from GHSA-7cc2-r658-7xpf
This fixes a vulnerability with the `CODER_OIDC_EMAIL_DOMAIN` option,
where users with a superset of the allowed email domain would be allowed
to login. For example, given `CODER_OIDC_EMAIL_DOMAIN=google.com`, a
user would be permitted entry if their email domain was
`colin-google.com`.
2024-03-04 12:52:03 -05:00
Garrett Delfosse 8f190b2016 fix: disallow out of range ports (#12414) 2024-03-04 12:25:06 -05:00
Kayla Washburn-Love 3a86ae569a refactor: use TableEmpty in user settings (#12389) 2024-03-04 09:45:40 -07:00
Mathias Fredriksson 4ce1448bbe fix(cli): generate correctly named file in DumpHandler (#12409) 2024-03-04 18:35:33 +02:00
Bruno Quaresma afcea74462 fix(site): retry and debug passing build parameters options (#12384) 2024-03-04 10:25:53 -03:00
Kayla Washburn-Love af4d0b148b chore: add stories for Popover (#12387) 2024-03-01 15:43:35 -07:00
Michael Brewer 722ff50e59 fix: add service banner to workspace page (#12381) 2024-03-01 10:53:03 -07:00
Kayla Washburn-Love 4f0b885c30 chore: add stories for UserAvatar (#12376) 2024-03-01 10:50:17 -07:00
Kayla Washburn-Love 7824bee25f chore: add stories for Stack (#12375) 2024-03-01 10:26:50 -07:00
Kayla Washburn-Love f4c888f33e chore: add stories for Latency component (#12374) 2024-03-01 10:26:38 -07:00
Kayla Washburn-Love f00935baa6 chore: add stories for TableEmpty and TableLoader (#12373) 2024-03-01 10:26:30 -07:00
Cian Johnston b1c2fea78b feat(cli): add support cmd (#12328)
Part of #12163

- Adds a command coder support bundle <workspace> that generates a 
  support bundle and writes it to coder-support-$(date +%s).zip.
- Note: this is hidden currently until the rest of the functionality is fleshed out.
2024-03-01 17:13:50 +00:00
Colin Adler e5d911462f fix(tailnet): enforce valid agent and client addresses (#12197)
This adds the ability for `TunnelAuth` to also authorize incoming wireguard node IPs, preventing agents from reporting anything other than their static IP generated from the agent ID.
2024-03-01 09:02:33 -06:00
Colin Adler 7fbca62e08 chore: fix Test_parseInsightsStartAndEndTime flake (#12377)
Fixes https://github.com/coder/coder/issues/10600
2024-02-29 18:20:25 -06:00
Stephen Kirby 5a53afda46 minor change to quiet hours docs (#12338) 2024-02-29 14:50:00 -06:00
Bruno Quaresma 26b483d95e fix(site): fix form layout for tablet viewports (#12369) 2024-02-29 16:24:06 -03:00
Steven Masley 4006974a98 fix: external auth device flow, check both queries for errors (#12367)
* fix: external auth device flow, check both queries for errors
* Minor style update

---------

Co-authored-by: BrunoQuaresma <bruno_nonato_quaresma@hotmail.com>
2024-02-29 13:00:16 -06:00
Cian Johnston 9f3591add8 chore(cli): use xerrors.Errorf instead of fmt.Errorf (#12368) 2024-02-29 18:58:48 +00:00
Steven Masley cbcf4ef2c4 chore: add faking 429 responses from fake idp (#12365)
Required to trigger error condition in fe.
See pull (#12367)
2024-02-29 09:45:53 -06:00
Cian Johnston eba8cd7c07 chore: consolidate various randomPort() implementations (#12362)
Consolidates our existing randomPort() implementations to package testutil
2024-02-29 12:51:44 +00:00
Cian Johnston 4f87ba46f9 chore: update provisioner tag documentation with suggestions from #12315 (#12347)
- Adds more testcases to TestAcquirer_MatchTags
- Adds functionality to generate a table from above test
- Update provisioner tag documentation with generated table
- Apply other feedback from #12315
2024-02-29 12:31:11 +00:00
Cian Johnston e57c101200 feat: add support package and accompanying tests (#12289) 2024-02-29 11:58:33 +00:00
Cian Johnston 2bf3c72948 chore: add test for enterprise server cli (#12353) 2024-02-29 10:25:50 +00:00
Cian Johnston b17fcd9cff ci: use linter version from Dockerfile (#12354) 2024-02-29 09:53:32 +00:00
Kayla Washburn-Love b24ad1bbf0 refactor: show parameter suggestions from user history below field (#12340) 2024-02-28 15:29:48 -07:00
Kyle Carberry b2a5e2f4c0 fix: Increase license key rows (#12352)
It was pretty hard to tell when you pasted something in
this box with only displaying a single line.

This should help!
2024-02-28 21:57:10 +00:00
Steven Masley 97f083810f chore: provide usage instruction for CLI argument failures (#12309)
* chore: add usage to # cli arg failures
2024-02-28 12:10:17 -06:00
Marcin Tojek 30d9d84758 fix: use flag to enable Prometheus (#12345) 2024-02-28 17:58:03 +01:00
Dean Sheather bedd2c5922 fix: avoid race between replicas on start (#12344)
DERP mesh key setup would do a SELECT and then an INSERT on failure, without a lock. During some testing with multiple replicas, I managed to cause a replica to crash due to them initializing simultaneously.

Fixes:

Encountered an error running "coder server"
create coder API: insert mesh key: pq: duplicate key value violates unique constraint "site_configs_key_key"

Co-authored-by: Cian Johnston <cian@coder.com>
2024-02-28 16:14:11 +00:00
Bruno Quaresma 76273bf369 feat(site): display client errors in DERP Region health page (#12318) 2024-02-28 13:30:38 +00:00
Cian Johnston 1465ee2ed1 fix(coderd): use database.IsQueryCanceledError instead of xerrors.Is(err, context.Canceled) (#12325) 2024-02-28 21:19:57 +10:00
Marcin Tojek eb4a1e2568 feat: enable Prometheus endpoint for external provisioner (#12320) 2024-02-28 09:21:56 +01:00
Michael Smith 087f973415 refactor(site): clean up clipboard functionality and define tests (#12296)
* refactor: clean up and update API for useClipboard

* wip: commit current progress on useClipboard test

* docs: clean up wording on showCopySuccess

* chore: make sure tests can differentiate between HTTP/HTTPS

* chore: add test ID to dummy input

* wip: commit progress on useClipboard test

* wip: commit more test progress

* refactor: rewrite code for clarity

* chore: finish clipboard tests

* fix: prevent double-firing for button click aliases

* refactor: clean up test setup

* fix: rename incorrect test file

* refactor: update code to display user errors

* refactor: redesign useClipboard to be easier to test

* refactor: clean up GlobalSnackbar

* feat: add functionality for notifying user of errors (with tests)

* refactor: clean up test code

* refactor: centralize cleanup steps
2024-02-27 21:05:37 -05:00
dependabot[bot] e183843a16 chore: bump google.golang.org/grpc from 1.61.0 to 1.62.0 (#12301)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-28 02:37:44 +05:00
Bruno Quaresma 0fc1a9164e feat(site): support zip upload for template files (#12323)
Related to #11687
2024-02-27 18:00:23 -03:00
Eric Paulsen 0f9c142ea6 docs: add k8s security reference (#12334)
* docs: add k8s security reference

* make fmt
2024-02-27 15:59:11 -05:00
Kayla Washburn-Love 30772b80c1 Revert "ci: bump the github-actions group with 1 update (#12303)" (#12327)
This reverts commit 5757321ba2.
2024-02-27 19:49:50 +00:00
Kayla Washburn-Love b2413a593c chore: reimplement activity status and autostop improvements (#12175) 2024-02-27 11:06:26 -07:00
Spike Curtis 4e7beee102 feat: show tailnet peer diagnostics after coder ping (#12314)
Beginnings of a solution to #12297 

Doesn't cover disco or definitively display whether we successfully connected to DERP, but shows some checklist diagnostics for connecting to an agent.

For this first PR, I just added it to `coder ping` to see how we like it, but could be incorporated into `coder ssh` _et al._ after a timeout.

```
$ coder ping dogfood2
p2p connection established in 147ms
pong from dogfood2 p2p via  95.217.xxx.yyy:42631  in 147ms
pong from dogfood2 p2p via  95.217.xxx.yyy:42631  in 140ms
pong from dogfood2 p2p via  95.217.xxx.yyy:42631  in 140ms
✔ preferred DERP region 999 (Council Bluffs, Iowa)
✔ sent local data to Coder networking coodinator
✔ received remote agent data from Coder networking coordinator
    preferred DERP 10013 (Europe Fly.io (Paris))
    endpoints: 95.217.xxx.yyy:42631, 95.217.xxx.yyy:37576, 172.17.0.1:37576, 172.20.0.10:37576
✔ Wireguard handshake 11s ago
```
2024-02-27 22:04:46 +04:00
Mathias Fredriksson 32691e67e6 test(agent/agentscripts): fix test flake in TestEnv (#12326) 2024-02-27 17:58:10 +00:00
Kayla Washburn-Love cbaf1c65ef chore: clean out site/out/assets/ when building to prevent "too much data" errors (#12313) 2024-02-27 10:45:57 -07:00
Cian Johnston b9e2d0a400 fix(coderd): mark provisioner daemon psk as secret (#12322)
* fix(coderd): mark provisioner daemon psk as secret

Marks provisioner daemon PSK with the secret annotation.
This ensures it will be scrubbed from API requests to
/api/v2/deployment/config.

* make gen
2024-02-27 16:33:32 +00:00
Steven Masley 19baca55da feat: implement create org commands from cli (#12308)
* feat: implement create org commands from cli
2024-02-27 10:13:08 -06:00
dependabot[bot] 5757321ba2 ci: bump the github-actions group with 1 update (#12303)
Bumps the github-actions group with 1 update: [chromaui/action](https://github.com/chromaui/action).


Updates `chromaui/action` from 10 to 11
- [Release notes](https://github.com/chromaui/action/releases)
- [Commits](https://github.com/chromaui/action/compare/v10...v11)

---
updated-dependencies:
- dependency-name: chromaui/action
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-27 11:53:48 -03:00
Cian Johnston 1d65e36b89 ci: increase connection hard limit for fly.io wsproxies (#12319) 2024-02-27 13:57:24 +00:00
Cian Johnston 392fecee87 chore(docs): update external provisioners documentation (#12315) 2024-02-27 13:05:28 +00:00
Cian Johnston 96c9838ce3 fix(cli): scaletest: do not screenshot if verbose=false (#12317) 2024-02-27 12:35:48 +00:00
Bruno Quaresma 2ca8248315 chore(site): apply code conventions (#12316) 2024-02-27 12:24:07 +00:00
dependabot[bot] 5a0d9db6c3 chore: bump github.com/elastic/go-sysinfo from 1.12.0 to 1.13.1 (#12213)
Bumps [github.com/elastic/go-sysinfo](https://github.com/elastic/go-sysinfo) from 1.12.0 to 1.13.1.
- [Release notes](https://github.com/elastic/go-sysinfo/releases)
- [Commits](https://github.com/elastic/go-sysinfo/compare/v1.12.0...v1.13.1)

---
updated-dependencies:
- dependency-name: github.com/elastic/go-sysinfo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-27 15:19:43 +05:00
Selina NN 431bf5cf3b Update artifactory-integration.md (#12311)
Update date year
2024-02-27 14:57:26 +05:00
Asher f74532ff50 feat: audit oauth2 app management (#12275)
* Audit oauth2 app management
* Use 201 for creating secrets
2024-02-26 23:52:08 +00:00
Steven Masley 6b866b3f48 feat: set sane default for gitea external auth (#12306)
* feat: external auth defaults for gitea

Add some sane defaults for gitea to make it easier to configure
2024-02-26 12:35:18 -06:00
Steven Masley 70ccefc357 feat: set organization context in coder organizations (#12265)
* feat: add coder organizations set to change org context

`coder organizations set <org>`
2024-02-26 11:39:26 -06:00
Steven Masley 748cf4b2c4 feat: implement global flag for org selection (#12276)
* feat: implement global flag for org selection

Any command can use '-z' to override org context
2024-02-26 11:38:49 -06:00
dependabot[bot] 5a41385400 chore: bump github.com/gohugoio/hugo from 0.122.0 to 0.123.3 (#12302)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-26 21:18:19 +05:00
Steven Masley d2998c6b7b feat: implement organization context in the cli (#12259)
* feat: implement organization context in the cli

`coder org show current`
2024-02-26 10:03:49 -06:00
Steven Masley f44c89d200 chore: enforce orgid in audit logs where required (#12283)
* chore: enforce orgid in audit logs where required
2024-02-26 08:27:33 -06:00
Cian Johnston 74b749b890 chore(coderd): add test to assert agent token invalid when workspace deleted (#12290) 2024-02-26 13:27:00 +00:00
Muhammad Atif Ali 7eed40bd99 chore(dogfood): bump jetbrains-gateway module to 1.0.6 (#12298) 2024-02-26 11:57:40 +00:00
Spike Curtis b0afffbafb feat: use v2 API for agent metadata updates (#12281)
Switches the agent to report metadata over the v2 API.

Fixes #10534
2024-02-26 09:50:19 +04:00
Gary Reynolds 7a245e61b1 chore(docs): inline OIDC flow diagram (#12255)
When viewing the Authentication page, the diagram showing the flow is a useful
resource for understanding the rest of the page.

Rather than linking to a specific version of the SVG, inline it as part of the
documentation.
2024-02-26 04:48:01 +00:00
Michael Brewer 245e280531 docs: add gitlab self-managed example (#12295) 2024-02-25 10:11:13 -05:00
Eric Paulsen fb198ac99c docs: add steps for postgres server verification (#12072)
* docs: add steps for postgres server verification

* make: fmt

* refactor to guide

* add manifest
2024-02-25 01:16:56 +00:00
dependabot[bot] 7e797e90ac chore: bump golang.org/x/tools from 0.17.0 to 0.18.0 (#12209)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-24 10:42:50 +05:00
Steven Masley c33c452663 fix: use default org over index [0] for new scim (#12284) 2024-02-23 15:31:36 -06:00
Colin Adler 0dd126e025 chore: upgrade github.com/hashicorp/hc-install to v0.6.3 (#12288)
Removes a dependency on a vulnerable version of github.com/cloudflare/circl
2024-02-23 13:41:58 -06:00
Bruno Quaresma 79480ca587 feat(site): display build logs on template creation (#12271) 2024-02-23 12:23:52 -07:00
Steven Masley 13359aa16f chore: drop github per user rate limit tracking (#12286)
* chore: drop github per user rate limit tracking

Rate limits for authenticated requests are per user.
This would be an excessive number of prometheus labels,
so we only track the unauthorized limit.
2024-02-23 11:17:52 -06:00
Marcin Tojek 90db6683c4 fix: refresh entitlements after creating first user (#12285) 2024-02-23 16:48:24 +00:00
Cian Johnston 2cb9bfd517 refactor(coderd): move healthcheck report structs to codersdk (#12279)
Moves healthcheck report-related structs from coderd/healthcheck to codersdk
This prevents an import cycle when adding a codersdk.Client method to hit /api/v2/debug/health.
2024-02-23 13:13:28 +00:00
Spike Curtis aa7a9f5cc4 feat: use v2 API for agent lifecycle updates (#12278)
Agent uses the v2 API to post lifecycle updates.

Part of #10534
2024-02-23 15:24:28 +04:00
Dean Sheather ee7828a166 chore: fix wsproxy test flake (#12280)
* chore: fix wsproxy test flake

* fixup! chore: fix wsproxy test flake
2024-02-23 21:19:54 +10:00
Spike Curtis 4cc132cea0 feat: switch agent to use v2 API for sending logs (#12068)
Changes the agent to use the new v2 API for sending logs, via the logSender component.

We keep the PatchLogs function around, but deprecate it so that we can test the v1 endpoint.
2024-02-23 11:27:15 +04:00
Spike Curtis af3fdc68c3 chore: refactor agent routines that use the v2 API (#12223)
In anticipation of needing the `LogSender` to run on a context that doesn't get immediately canceled when you `Close()` the agent, I've undertaken a little refactor to manage the goroutines that get run against the Tailnet and Agent API connection.

This handles controlling two contexts, one that gets canceled right away at the start of graceful shutdown, and another that stays up to allow graceful shutdown to complete.
2024-02-23 11:04:23 +04:00
Kayla Washburn-Love 66585f042f feat: support markdown in update messages (#12273) 2024-02-22 16:14:06 -07:00
Kayla Washburn-Love 7e6cb66a50 feat(site): allow creating a workspace without connecting optional external auth providers (#12251) 2024-02-22 10:27:36 -07:00
Kayla Washburn-Love b8a53230c7 chore: revert "refactor(site): verify external auth before display ws form (#11777)" (#12183) 2024-02-22 09:44:30 -07:00
Cian Johnston 53e8f9c0f9 fix(coderd): only allow untagged provisioners to pick up untagged jobs (#12269)
Alternative solution to #6442

Modifies the behaviour of AcquireProvisionerJob and adds a special case for 'un-tagged' jobs such that they can only be picked up by 'un-tagged' provisioners.

Also adds comprehensive test coverage for AcquireJob given various combinations of tags.
2024-02-22 15:04:31 +00:00
Marcin Tojek aa7a12a5ec docs: document Terraform variables (#12270) 2024-02-22 15:26:53 +01:00
Steven Masley d4d8424ce0 fix: fix GetOrganizationsByUserID error when multiple organizations exist (#12257)
* test: fetching user orgs fails if multi orgs in pg db
* fix: GetOrganizationsByUserID fixed if multi orgs exist
2024-02-22 08:14:48 -06:00
Spike Curtis da376549a3 fix: stop waiting for Agent in a goroutine in ssh test (#12268)
Fixes race seen here: https://github.com/coder/coder/runs/21852483781

What happens is that the agent connects, completes the test, and then disconnects before the Eventually condition runs.  The waiter then times out because it's looking for a connected agent.

Then, since it's a `require` in a goroutine, that causes the `tGo` cleanup to hang and the whole test suite to timeout after 10 minutes.

Anyway, `agenttest.New` doesn't block, and we don't actually need to wait for the agent to connect, since a successful SSH session is evidence that it connected.
2024-02-22 17:01:06 +04:00
dependabot[bot] a31a05e2cb chore: bump github.com/valyala/fasthttp from 1.51.0 to 1.52.0 (#12210)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-22 11:59:29 +05:00
dependabot[bot] 307a206605 chore: bump github.com/prometheus/client_model from 0.5.0 to 0.6.0 (#12212)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-22 11:59:17 +05:00
Asher 51d178d538 feat: add OAuth2 user settings page (#12237) 2024-02-21 14:16:55 -09:00
dependabot[bot] 3cbe14fdad chore: bump ip from 2.0.0 to 2.0.1 in /site (#12238)
Bumps [ip](https://github.com/indutny/node-ip) from 2.0.0 to 2.0.1.
- [Commits](https://github.com/indutny/node-ip/compare/v2.0.0...v2.0.1)

---
updated-dependencies:
- dependency-name: ip
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 14:06:43 -09:00
Asher 7eb2beccea fix: redirect from oauth2 authorization page (#12241)
At the moment it just says "you are not authorized", but we want to
automatically redirect to the login page.
2024-02-21 13:30:33 -09:00
Steven Masley c3a7b13690 chore: remove organization requirement from convertGroup() (#12195)
* feat: convertGroups() no longer requires organization info

Removing role information from some users in the api. This info is
excessive and not required. It is costly to always include
2024-02-21 15:58:11 -06:00
Steven Masley 3f65bd14cc fix: ignore surronding whitespace for cli config (#12250)
* fix: ignore surronding whitespace for cli config

Cli config files break if you edit them manually with any editor.
Editors drop a newline at the end, and we not break on this.
If a developer manually edits a file, it should still work
2024-02-21 13:03:41 -06:00
Kayla Washburn-Love 475c3650ca feat: add support for optional external auth providers (#12021) 2024-02-21 11:18:38 -07:00
Bruno Quaresma 78c9f82719 fix(site): fix error when typing long number on ttl (#12249) 2024-02-21 17:29:29 +00:00
Michael Smith 1d254f4680 fix: add tests and improve accessibility for useClickable (#12218) 2024-02-21 10:59:13 -05:00
Bruno Quaresma a827185b6d refactor: move auto fill feature into an experiment (#12230) 2024-02-21 11:48:34 -03:00
Marcin Tojek c230bcf5ca fix: previous parameter value is not a number (#12246) 2024-02-21 15:44:45 +01:00
Bruno Quaresma b4fb754b2d feat(site): show previous agent scripts logs (#12233) 2024-02-21 11:42:34 -03:00
Bruno Quaresma 0398e3c531 chore(site): fix storybook for agent row with port forward button (#12247) 2024-02-21 14:32:39 +00:00
Bruno Quaresma cc4cefbbee chore(site): fix storybook test (#12245) 2024-02-21 14:17:18 +00:00
Bruno Quaresma ebe05820c9 fix(site): fix web terminal bottom overflow (#12228) 2024-02-21 11:02:53 -03:00
dependabot[bot] 91c3df785f chore: bump github.com/aws/smithy-go from 1.19.0 to 1.20.0 (#12206)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 13:09:18 +05:00
dependabot[bot] 519cf5935f chore: bump github.com/prometheus/common from 0.46.0 to 0.47.0 (#12207)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 11:36:52 +05:00
Asher 3d742f64e6 fix: move oauth2 routes (#12240)
* fix: move oauth2 routes

From /login/oauth2/* to /oauth2/*.

/login/oauth2 causes /login to no longer get served by the frontend,
even if nothing is actually served on /login itself.

* Add forgotten comment on delete
2024-02-20 17:01:25 -09:00
Asher 4d39da294e feat: add oauth2 token exchange (#12196)
Co-authored-by: Steven Masley <stevenmasley@gmail.com>
2024-02-20 14:58:43 -09:00
Steven Masley 07cccf9033 feat: disable directory listings for static files (#12229)
* feat: disable directory listings for static files

Static file server handles serving static asset files (js, css, etc).
The default file server would also list all files in a directory.
This has been changed to only serve files.
2024-02-20 15:50:30 -06:00
Steven Masley 2dac34276a fix: add postgres triggers to remove deleted users from user_links (#12117)
* chore: add database test fixture to insert non-unique linked_ids
* chore: create unit test to exercise failed email change bug
* fix: add postgres triggers to keep user_links clear of deleted users
* Add migrations to prevent deleted users with links
* Force soft delete of users, do not allow un-delete
2024-02-20 13:19:38 -06:00
Garrett Delfosse b342bd7869 feat: add port sharing frontend (#12119) 2024-02-20 13:26:34 -05:00
Bruno Quaresma 0021c2f906 fix(site): fix parameters field size (#12231) 2024-02-20 13:54:07 -03:00
Marcin Tojek 57bf997369 feat: support custom validation errors for number-typed parameters (#12224) 2024-02-20 16:32:03 +01:00
Michael Smith 6414b7aade chore(site): refactor tests for global hooks   (#12216)
* refactor: clean up tests for debounce

* refactor: clean up tests for useCustomEvent

* refactor: clean up events file

* refactor: clean up tests for hookPolyfills
2024-02-20 09:19:43 -05:00
Michael Smith d6ae9d8548 revert: remove anti-flicker clipboard styling (#12227)
- These CSS changes were for making sure there weren't layout shifts
  when using the non-secure clipboard fallback, which could cause janky
  UI flickers. It seems to be breaking things for some users on HTTP-only
  connections, though.
2024-02-20 14:14:25 +00:00
Cian Johnston 643c3ee54b refactor(provisionerd): move provisionersdk.VersionCurrent -> provisionerdproto.VersionCurrent (#12225) 2024-02-20 12:44:19 +00:00
Cian Johnston c62a8b0bee fix(helm)!: remove prometheus-http port declaration from coderd service spec (#12214)
This PR removes the prometheus-http port entirely from the coder service specification (originally added in #10448). It also removes the Helm value coder.service.prometheusNodePort.

Rationale: some cloud providers will helpfully expose all ports on a LoadBalancer service for you. The net effect of this is that setting CODER_PROMETHEUS_ENABLE will end up exposing port 2112 on your coderd service to the internet, which is likely undesired behaviour.
2024-02-20 11:36:17 +00:00
Mathias Fredriksson b1c0b39d88 feat(agent): add script data dir for binaries and files (#12205)
The agent is extended with a `--script-data-dir` flag, defaulting to the
OS temp dir. This dir is used for storing `coder-script-data/bin` and
`coder-script/[script uuid]`. The former is a place for all scripts to
place executable binaries that will be available by other scripts, SSH
sessions, etc. The latter is a place for the script to store files.

Since we default to OS temp dir, files are ephemeral by default. In the
future, we may consider adding new env vars or changing the default
storage location. Workspace startup speed could potentially benefit from
scripts being able to skip steps that require downloading software. We
may also extend this with more env variables (e.g. persistent storage in
HOME).

Fixes #11131
2024-02-20 13:26:18 +02:00
Spike Curtis ab4cb66e00 feat: add WaitUntilEmpty to LogSender (#12159)
We'll need this to be able to tell when all outstanding logs have been sent, as part of graceful shutdown.
2024-02-20 11:11:31 +04:00
Spike Curtis 081e37d7d9 chore: move LogSender to agentsdk (#12158)
Moves the LogSender to agentsdk and deprecates LogsSender based on the v1 API.
2024-02-20 10:44:20 +04:00
Dean Sheather 9861830e87 fix: never send local endpoints if disabled (#12138) 2024-02-20 15:51:25 +10:00
Mathias Fredriksson c63f569174 refactor(agent/agentssh): move envs to agent and add agentssh config struct (#12204)
This commit refactors where custom environment variables are set in the
workspace and decouples agent specific configs from the `agentssh.Server`.
To reproduce all functionality, `agentssh.Config` is introduced.

The custom environment variables are now configured in `agent/agent.go`
and the agent retains control of the final state. This will allow for
easier extension in the future and keep other modules decoupled.
2024-02-19 16:30:00 +02:00
Colin Adler 817cc78b94 fix(examples): remove dead code comment (#12194) 2024-02-17 17:38:19 +00:00
Mathias Fredriksson 0442ee5fa8 fix(agent/reconnectingpty): fix screen startup speed by disabling messages (#12190) 2024-02-16 22:37:02 +02:00
Cian Johnston a2cbb0f87f fix(enterprise/coderd): check provisionerd API version on connection (#12191) 2024-02-16 18:43:07 +00:00
Steven Masley f17149c59d feat: set groupsync to use default org (#12146)
* fix: assign new oauth users to default org

This is not a final solution, as we eventually want to be able
to map to different orgs. This makes it so multi-org does not break oauth/oidc.
2024-02-16 11:09:19 -06:00
Kayla Washburn-Love dbaafc863c chore: update no-restricted-imports lint rule (#12180)
- prevent importing from the "monolith" lodash module. individual modules are better for tree shaking.
- prevent importing `useTheme` and types from @mui/material/styles. prefer importing from @emotion/react.
2024-02-16 09:54:40 -07:00
Steven Masley 75870c22ab fix: assign new oauth users to default org (#12145)
* fix: assign new oauth users to default org

This is not a final solution, as we eventually want to be able
to map to different orgs. This makes it so multi-org does not break oauth/oidc.
2024-02-16 08:47:26 -06:00
Steven Masley 2a8004b1b2 feat: use default org for PostUser (#12143)
Instead of assuming only 1 org exists, this uses the
is_default org to place a user in if not specified.
2024-02-16 08:28:36 -06:00
Marcin Tojek 0e1bad4f82 docs: fix header font (#12193) 2024-02-16 13:32:45 +00:00
Muhammad Atif Ali 799d71f6b2 docs: simplify docker installation docs (#12187) 2024-02-16 12:53:03 +00:00
Bruno Quaresma be1edc3995 fix(site): fix language detection for Dockerfile (#12188) 2024-02-16 12:50:36 +00:00
Marcin Tojek 41647ca984 docs: describe resource ordering in UI (#12185) 2024-02-16 13:33:57 +01:00
Bruno Quaresma df297627c2 fix(site): match activity bump text with template settings (#12170)
Close https://github.com/coder/coder/issues/12130
2024-02-16 09:33:15 -03:00
Muhammad Atif Ali 99dbeb4a85 ci: fix broken dogfood workflow (#12186) 2024-02-16 14:14:35 +03:00
Muhammad Atif Ali 8ca2add6dc chore(dogfood): revert to pre-artifactory state (#12169) 2024-02-16 13:47:15 +03:00
Colin Adler 97e4d51953 fix(cli/clibase): don't error on required flags with --help (#12181) 2024-02-15 23:41:46 +00:00
Michael Smith fbd436cc2c fix: improve clipboard support on HTTP connections and older browsers (#12178)
* fix: add future-proofing for clipboard copies on http connections

* docs: clean up comment formatting
2024-02-15 16:44:53 -05:00
Colin Adler 8a9f59a4bb fix(cli): avoid panic when external auth name isn't provided (#12177)
Fixes https://github.com/coder/coder/issues/10216
2024-02-15 15:17:16 -06:00
Colin Adler 4c3d44658d fix(codersdk): correctly log coordination error (#12176) 2024-02-15 20:47:12 +00:00
Steven Masley 2bf2f88b09 feat: implement 'is_default' org field (#12142)
The first organization created is now marked as "default". This is
to allow "single org" behavior as we move to a multi org codebase.

It is intentional that the user cannot change the default org at this
stage. Only 1 default org can exist, and it is always the first org.

Closes: https://github.com/coder/coder/issues/11961
2024-02-15 11:01:16 -06:00
dependabot[bot] a67362fdb1 chore: bump github.com/u-root/u-root from 0.12.0 to 0.13.0 (#12100)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-15 19:25:52 +03:00
Marcin Tojek 5aa5ff1bde chore: deprecate API workspace build resources (#12167) 2024-02-15 17:13:44 +01:00
Spike Curtis 2aff014e5d feat: add logSender for sending logs on agent v2 API (#12046)
Adds a new subcomponent of the agent for queueing up logs until they can be sent over the Agent API.

Subsequent PR will change the agent to use this instead of the HTTP API for posting logs.

Relates to #10534
2024-02-15 16:57:17 +04:00
Spike Curtis 627232eae9 fix: fix pgcoord to delete coordinator row last (#12155)
Fixes #12141
Fixes #11750

PGCoord shutdown was uncoordinated, so an update at an inopportune time during shutdown would be rejected because the coordinator row was already deleted.

This PR ensures that the PGCoord subcomponents that write updates are shut down before we take down the heartbeats, which is responsible for deleting the coordinator row.
2024-02-15 16:34:29 +04:00
Marcin Tojek 7a453608c9 feat: support order property of coder_agent (#12121) 2024-02-15 13:33:13 +01:00
Sulochan c66e665864 docs: add kubevirt coder template in list of community templates (#12113) 2024-02-15 13:18:10 +03:00
Marcin Tojek 8cc62fb221 fix(site): ignore fileInfo if file is missing (#12154) 2024-02-15 09:15:22 +00:00
Muhammad Atif Ali d9f99da327 chore(docs): update artifactory-integration guide (#12153) 2024-02-15 11:20:50 +03:00
Spike Curtis 2d0b9106c0 fix: change servertailnet to register the DERP dialer before setting DERP map (#12137)
I noticed a possible race where tailnet.Conn can try to dial the embedded region before we've set our custom dialer that send the DERP in-memory.  This closes that race and adds a test case for servertailnet with no STUN and an embedded relay
2024-02-15 10:51:12 +04:00
dependabot[bot] 1bb4aecf49 chore: bump golang.org/x/oauth2 from 0.16.0 to 0.17.0 (#12099)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.16.0 to 0.17.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-14 13:37:30 -06:00
Cian Johnston d6b025db14 Revert "feat: add activity status and autostop reason to workspace overview (#11987)" (#12144)
Related to https://github.com/coder/coder/pull/11987

This reverts commit d37b131.
2024-02-14 17:14:49 +00:00
Spike Curtis 04991f425a fix: set node callback each time we reinit the coordinator in servertailnet (#12140)
I think this will resolve #12136 but lets get a proper test at the system level before closing.

Before this change, we only register the node callback at start of day for the server tailnet.  If the coordinator changes, like we know happens when we are licensed for the PGCoordinator, we close the connection to the old coord, and open a new one to the new coord.

The callback is designed to direct the updates to the new coordinator, but there is nothing that specifically triggers it to fire after we connect to the new coordinator.

If we have STUN, then period re-STUNs will generally get it to fire eventually, but without STUN it we could go indefinitely without a callback.

This PR changes the servertailnet to re-register the callback each time we reconnect to the coordinator.  Registering a callback (even if it's the same callback) triggers an immediate call with our node information, so the new coordinator will have it.
2024-02-14 20:45:31 +04:00
Spike Curtis 5a0d240bc3 feat: expose DERP server debug metrics (#12135)
Adds some debug endpoints for looking into the DERP server.

The `api/v2/debug/derp/traffic` endpoint requires the `ss` utility to be present in order to function.  I have *not* added the `iproute2` package to our base image as it adds 11MB, so this endpoint won't be useful by default.  However, in a debugging situation, we could exec into the container and then `apk add iproute2`, or build a special debug image.

The `api/v2/debug/expvar` handler contains DERP metrics as well as commandline and memstats.

Example:

```
{
"alert_failed": 0,
"alert_generated": 0,
"cmdline": ["/Users/spike/repos/coder/build/coder_darwin_arm64","--global-config","/Users/spike/repos/coder/.coderv2","server","--http-address","0.0.0.0:3000","--swagger-enable","--access-url","http://127.0.0.1:3000","--dangerous-allow-cors-requests=true"],
"derp": {"accepts": 1, "average_queue_duration_ms": 0, "bytes_received": 0, "bytes_sent": 0, "counter_packets_dropped_reason": {"gone_disconnected": 0, "gone_not_here": 0, "queue_head": 0, "queue_tail": 0, "unknown_dest": 0, "unknown_dest_on_fwd": 0, "write_error": 0}, "counter_packets_dropped_type": {"disco": 0, "other": 0}, "counter_packets_received_kind": {"disco": 0, "other": 0}, "counter_tcp_rtt": {}, "counter_total_dup_client_conns": 0, "gauge_clients_local": 1, "gauge_clients_remote": 0, "gauge_clients_total": 1, "gauge_current_connections": 1, "gauge_current_dup_client_conns": 0, "gauge_current_dup_client_keys": 0, "gauge_current_file_descriptors": 0, "gauge_current_home_connections": 1, "gauge_memstats_sys0": 20874504, "gauge_watchers": 0, "got_ping": 0, "home_moves_in": 0, "home_moves_out": 0, "multiforwarder_created": 0, "multiforwarder_deleted": 0, "packet_forwarder_delete_other_value": 0, "packets_dropped": 0, "packets_forwarded_in": 0, "packets_forwarded_out": 0, "packets_received": 0, "packets_sent": 0, "peer_gone_disconnected_frames": 0, "peer_gone_not_here_frames": 0, "sent_pong": 0, "unknown_frames": 0, "version": "1.47.0-dev20240214-t64db8c604"},
"memstats": {"Alloc":286506256,"TotalAlloc":297594632,"Sys":310621512,"Lookups":0,"Mallocs":304204,"Frees":171570,"HeapAlloc":286506256,"HeapSys":294060032,"HeapIdle":3694592,"HeapInuse":290365440,"HeapReleased":3620864,"HeapObjects":132634,"StackInuse":3735552,"StackSys":3735552,"MSpanInuse":347256,"MSpanSys":358512,"MCacheInuse":9600,"MCacheSys":15600,"BuckHashSys":1469877,"GCSys":9434896,"OtherSys":1547043,"NextGC":551867656,"LastGC":1707892877408883000,"PauseTotalNs":1247000,"PauseNs":[200333,229375,239875,209542,106958,203792,57125,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"PauseEnd":[1707892876217481000,1707892876219726000,1707892876222273000,1707892876226151000,1707892876234815000,1707892877398146000,1707892877408883000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"NumGC":7,"NumForcedGC":0,"GCCPUFraction":0.0022425810335762954,"EnableGC":true,"DebugGC":false,"BySize":[{"Size":0,"Mallocs":0,"Frees":0},{"Size":8,"Mallocs":14396,"Frees":9143},{"Size":16,"Mallocs":89090,"Frees":50507},{"Size":24,"Mallocs":40839,"Frees":24456},{"Size":32,"Mallocs":22404,"Frees":12379},{"Size":48,"Mallocs":51174,"Frees":23718},{"Size":64,"Mallocs":15406,"Frees":3501},{"Size":80,"Mallocs":6688,"Frees":2352},{"Size":96,"Mallocs":2567,"Frees":374},{"Size":112,"Mallocs":19371,"Frees":16883},{"Size":128,"Mallocs":2873,"Frees":1061},{"Size":144,"Mallocs":5600,"Frees":2742},{"Size":160,"Mallocs":2159,"Frees":622},{"Size":176,"Mallocs":454,"Frees":86},{"Size":192,"Mallocs":227,"Frees":128},{"Size":208,"Mallocs":1407,"Frees":732},{"Size":224,"Mallocs":1365,"Frees":1090},{"Size":240,"Mallocs":82,"Frees":48},{"Size":256,"Mallocs":310,"Frees":162},{"Size":288,"Mallocs":1945,"Frees":562},{"Size":320,"Mallocs":1200,"Frees":458},{"Size":352,"Mallocs":133,"Frees":33},{"Size":384,"Mallocs":582,"Frees":51},{"Size":416,"Mallocs":747,"Frees":200},{"Size":448,"Mallocs":113,"Frees":22},{"Size":480,"Mallocs":34,"Frees":21},{"Size":512,"Mallocs":951,"Frees":91},{"Size":576,"Mallocs":364,"Frees":122},{"Size":640,"Mallocs":532,"Frees":270},{"Size":704,"Mallocs":93,"Frees":39},{"Size":768,"Mallocs":83,"Frees":35},{"Size":896,"Mallocs":308,"Frees":175},{"Size":1024,"Mallocs":226,"Frees":122},{"Size":1152,"Mallocs":198,"Frees":100},{"Size":1280,"Mallocs":314,"Frees":171},{"Size":1408,"Mallocs":77,"Frees":47},{"Size":1536,"Mallocs":80,"Frees":54},{"Size":1792,"Mallocs":199,"Frees":107},{"Size":2048,"Mallocs":112,"Frees":48},{"Size":2304,"Mallocs":71,"Frees":32},{"Size":2688,"Mallocs":206,"Frees":81},{"Size":3072,"Mallocs":39,"Frees":15},{"Size":3200,"Mallocs":16,"Frees":7},{"Size":3456,"Mallocs":44,"Frees":29},{"Size":4096,"Mallocs":192,"Frees":83},{"Size":4864,"Mallocs":44,"Frees":25},{"Size":5376,"Mallocs":105,"Frees":43},{"Size":6144,"Mallocs":25,"Frees":5},{"Size":6528,"Mallocs":22,"Frees":7},{"Size":6784,"Mallocs":3,"Frees":0},{"Size":6912,"Mallocs":4,"Frees":2},{"Size":8192,"Mallocs":59,"Frees":10},{"Size":9472,"Mallocs":31,"Frees":12},{"Size":9728,"Mallocs":5,"Frees":2},{"Size":10240,"Mallocs":5,"Frees":0},{"Size":10880,"Mallocs":27,"Frees":11},{"Size":12288,"Mallocs":4,"Frees":1},{"Size":13568,"Mallocs":4,"Frees":2},{"Size":14336,"Mallocs":9,"Frees":2},{"Size":16384,"Mallocs":10,"Frees":2},{"Size":18432,"Mallocs":4,"Frees":2}]},
"warning_failed": 0,
"warning_generated": 0
}
```

If we find the DERP metrics useful we could consider how to include them in Prometheus scrapes based on the tailnet `varz` package.  That's for a later PR if at all.
2024-02-14 15:11:45 +04:00
Muhammad Atif Ali 53c55439be chore (examples/templates/incus): fix a typo (#12123) 2024-02-13 19:16:33 +00:00
Steven Masley 5d483a7ea1 fix: do not query user_link for deleted accounts (#12112) 2024-02-13 13:02:21 -06:00
Steven Masley 06f3ab1206 chore: add database test fixture to insert non-unique linked_ids (#12111)
* chore: add database test fixture to insert non-unique linked_ids
2024-02-13 12:06:47 -06:00
Kayla Washburn-Love d37b131426 feat: add activity status and autostop reason to workspace overview (#11987) 2024-02-13 10:50:17 -07:00
Muhammad Atif Ali e53d8bdb50 docs: update modules docs (#11911) 2024-02-13 15:35:09 +00:00
Cian Johnston 68641f9e2f chore(examples/templates/incus): fix incus group name in README (#12120) 2024-02-13 15:31:07 +00:00
dependabot[bot] e938690b1e chore: bump golang.org/x/mod from 0.14.0 to 0.15.0 (#12094)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-13 18:25:26 +03:00
Muhammad Danish 3c536aa880 ci: use repo secret for syncing winget-pkgs fork (#12108) 2024-02-13 18:25:13 +03:00
dependabot[bot] 28bbdee655 chore: bump github.com/go-playground/validator/v10 (#12096)
Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.17.0 to 10.18.0.
- [Release notes](https://github.com/go-playground/validator/releases)
- [Commits](https://github.com/go-playground/validator/compare/v10.17.0...v10.18.0)

---
updated-dependencies:
- dependency-name: github.com/go-playground/validator/v10
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-13 17:54:13 +03:00
dependabot[bot] 4760e85c15 chore: bump golang.org/x/net from 0.20.0 to 0.21.0 (#12097)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.21.0.
- [Commits](https://github.com/golang/net/compare/v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-13 17:53:16 +03:00
dependabot[bot] 9560d9a68b ci: bump the github-actions group with 2 updates (#12091)
Bumps the github-actions group with 2 updates: [crate-ci/typos](https://github.com/crate-ci/typos) and [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action).


Updates `crate-ci/typos` from 1.18.0 to 1.18.2
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.18.0...v1.18.2)

Updates `aquasecurity/trivy-action` from 0.16.1 to 0.17.0
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](https://github.com/aquasecurity/trivy-action/compare/d43c1f16c00cfd3978dde6c07f4bbcf9eb6993ca...84384bd6e777ef152729993b8145ea352e9dd3ef)

---
updated-dependencies:
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: aquasecurity/trivy-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-13 17:52:12 +03:00
Garrett Delfosse 3ab3a62bef feat: add port-sharing backend (#11939) 2024-02-13 09:31:20 -05:00
Cian Johnston c939416702 chore(examples): add sample Incus template (#12114)
Adds sample incus template created for FOSDEM 2024; there's enough intricacy involved to make it worth persisting
2024-02-13 14:30:31 +00:00
Dean Sheather e1e352d8c1 feat: add template activity_bump property (#11734)
Allows template admins to configure the activity bump duration. Defaults to 1h.
2024-02-13 07:00:35 +00:00
Dean Sheather fead57f304 fix: allow access to unhealthy/initializing apps (#12086) 2024-02-13 16:30:49 +10:00
Cian Johnston ec25fb8bbc fix(docs/networking/stun): convert svg diagrams to png 2024-02-12 17:27:53 +00:00
Cian Johnston 2fabc9499a fix(docs): remove inline mermaid diagrams (#12107) 2024-02-12 15:56:37 +00:00
Cian Johnston 1cc51b009a chore(examples): remove deprecated startup_script_timeout and shutdown_script_timeout (#12104)
Removes deprecated startup_script_timeout and shutdown_script_timeout from our example templates.

Co-authored-by: Muhammad Atif Ali <atif@coder.com>
2024-02-12 14:29:41 +00:00
Marcin Tojek 3e68650791 feat: support order property of coder_app resource (#12077) 2024-02-12 15:11:31 +01:00
Cian Johnston 1e9a3c952f chore(docs/networking/stun): fix diagram in section 2 (#12103) 2024-02-12 12:33:41 +00:00
Cian Johnston d1a522a8fc chore(docs): add requirements re ports and stun server to docs (#12026)
Adds documentation on port requirements and a short overview of STUN with some example scenarios.

Co-authored-by: Dean Sheather <dean@deansheather.com>
Co-authored-by: Spike Curtis <spike@coder.com>
2024-02-12 11:42:27 +00:00
Dean Sheather 2fc3064653 chore: add tests for app ID copy in app healths (#12088) 2024-02-12 05:49:48 +00:00
Colin Adler 06254a167f chore(docs): add v2.8.2 changelog (#12089) 2024-02-12 05:48:34 +00:00
Dean Sheather 429144da22 fix: copy app ID in healthcheck (#12087) 2024-02-12 05:01:16 +00:00
Eric Paulsen bb308851f5 docs: fix jetbrains reconnect faq (#12073)
* docs: fix jetbrains reconnect faq

* make: fmt

* add asher feedback
2024-02-09 23:44:33 +00:00
Bruno Quaresma 390217b396 feat(site): add create template from scratch (#12082) 2024-02-09 14:42:26 +00:00
Cian Johnston 2b307c7c4e fix(cli/server): do not redirect /healthz (#12080) 2024-02-09 13:44:47 +00:00
Spike Curtis 92b2e26a48 feat: send log limit exceeded in response, not error (#12078)
When we exceed the db-imposed limit of logs, we need to communicate that back to the agent.  In v1 we did it with a 4xx-level HTTP status, but with dRPC, the errors are delivered as strings, which feels fragile to me for something we want to gracefully handle.

So, this PR adds the log limit exceeded as a field on the response message, and fixes the API handler to set it as appropriate instead of an error.
2024-02-09 16:17:20 +04:00
Spike Curtis 1f5a6d59ba chore: consolidate websocketNetConn implementations (#12065)
Consolidates websocketNetConn from multiple packages in favor of a central one in codersdk
2024-02-09 11:39:08 +04:00
Colin Adler ec8e41f516 chore: add logging around agent app health reporting (#12071) 2024-02-08 23:37:44 -06:00
Marcin Tojek c0e169ebf9 feat: support custom order of agent metadata (#12066) 2024-02-08 17:29:34 +01:00
Mathias Fredriksson e659957b65 fix(cli/ssh): prevent reads/writes to stdin/stdout in stdio mode (#12045)
Fixes #11530
2024-02-08 13:09:42 +02:00
Spike Curtis 151aaadc23 fix: allow startup scripts larger than 32k (#12060)
Fixes #12057 and adds a regression test.
2024-02-07 22:26:42 +04:00
Bruno Quaresma 4d63a473b2 fix(site): fix infinity loading when template has no previous version (#12059) 2024-02-07 14:56:09 -03:00
Mathias Fredriksson 040ce40ed8 fix(dogfood): add ability to synchronize with startup script via done file (#12058) 2024-02-07 19:16:18 +02:00
Bruno Quaresma d8a8070986 fix(site): enable submit when auto start and stop are both disabled (#12055) 2024-02-07 14:06:48 -03:00
Bruno Quaresma 4b1bac31b6 feat(site): allow any file extension on template editor (#12000) 2024-02-07 13:24:28 -03:00
Marcin Tojek 4e7b208068 fix(site): e2e: print API backend calls (#12051) 2024-02-07 15:50:07 +01:00
Eric Paulsen 1abe0cfa1a docs: fix /audit & /insights params (#12043) 2024-02-07 08:38:54 -05:00
Spike Curtis 1cf4b62867 feat: change agent to use v2 API for reporting stats (#12024)
Modifies the agent to use the v2 API to report its statistics, using the `statsReporter` subcomponent.
2024-02-07 15:26:41 +04:00
Muhammad Atif Ali 70ad833b02 ci: fix GH_TOKEN in release.yaml (#12044) 2024-02-07 13:37:11 +03:00
Mathias Fredriksson f2aef0726b fix(agent/agentssh): allow scp to exit with zero status (#12028)
Fixes #11786
2024-02-07 10:22:31 +02:00
Josh Vawdrey d3ccb07361 feat(cli): support header and header-command in config-ssh (#10413)
Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2024-02-07 10:21:26 +02:00
Ben Potter d6cdaae8b1 docs: add v2.8.0 changelog (#12042)
* docs: add v2.8.0 changelog

* fmt
2024-02-07 00:14:17 +00:00
Cian Johnston 36808f19dc feat!: update terraform to version 1.6.x, relax max version constraint (#12027)
* feat(provisioner): relax max terraform version constraint

* feat!(scripts/Dockerfile.base): update bundled terraform to 1.6.x

* bump terraform version in Dogfood image

* fix over-zealous rename
2024-02-06 17:58:26 -06:00
Kayla Washburn-Love b8e32a37de fix: use replace when redirecting from /health (#12039)
`pushHistory` will break the back button, so we need to use `replaceHistory` instead
2024-02-06 14:27:32 -07:00
Marcin Tojek 3f04e98cfa feat(cli): pull templates in zip format (#12032) 2024-02-06 19:17:29 +01:00
Spike Curtis 213ae69bee fix: start timer before subscribing to avoid test race (#12031)
Fixes #12030

This is a good example of the kind of thing I'd like to address with a time-testing lib.  The problem is that there is a race between the watchdog starting it's timer and the test incrementing the time.  What would make this easier is if the time-testing library could wait for and assert the call to start the timer before incrementing the time.
2024-02-06 20:21:23 +04:00
Marcin Tojek b6806bca70 fix: nix: google-chrome installed conditionally (#12029) 2024-02-06 16:46:58 +01:00
Dean Sheather 98b86f3cd6 chore: add logs to pq notification dialer (#12020) 2024-02-06 15:21:48 +00:00
Spike Curtis e09cd2c6bd feat: add watchdog to pubsub (#12011)
adds a watchdog to our pubsub and runs it for Coder server.

If the watchdog times out, it triggers a graceful exit in `coder server` to give any provisioner jobs a chance to shut down.

c.f. #11950
2024-02-06 16:58:45 +04:00
Cian Johnston f1e5b4fbb8 ci: stop deploying legacy wsproxies (#12025) 2024-02-06 11:00:10 +00:00
Cian Johnston 26379877b2 fix(dogfood): stop overriding /etc/apt/sources.list with tsw mirrors (#11999) 2024-02-06 09:39:05 +00:00
Colin Adler c7f52b73bb feat(coderd): add prometheus metrics to servertailnet (#11988) 2024-02-05 23:57:18 -06:00
Spike Curtis c84a637116 fix: stop logging error on query canceled (#12017)
Fixes flake seen here: https://github.com/coder/coder/actions/runs/7782340530/job/21218566449
2024-02-06 08:43:34 +04:00
Kayla Washburn-Love b73e66e9a9 feat: show workspace name suggestions below the name field (#12001) 2024-02-05 10:40:15 -07:00
dependabot[bot] 52ec3edd5d ci: bump the github-actions group with 4 updates (#12019)
Bumps the github-actions group with 4 updates: [buildjet/cache](https://github.com/buildjet/cache), [crate-ci/typos](https://github.com/crate-ci/typos), [codecov/codecov-action](https://github.com/codecov/codecov-action) and [hmarr/auto-approve-action](https://github.com/hmarr/auto-approve-action).


Updates `buildjet/cache` from 3 to 4
- [Changelog](https://github.com/BuildJet/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/buildjet/cache/compare/v3...v4)

Updates `crate-ci/typos` from 1.17.2 to 1.18.0
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.17.2...v1.18.0)

Updates `codecov/codecov-action` from 3 to 4
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v3...v4)

Updates `hmarr/auto-approve-action` from 3 to 4
- [Release notes](https://github.com/hmarr/auto-approve-action/releases)
- [Commits](https://github.com/hmarr/auto-approve-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: buildjet/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: hmarr/auto-approve-action
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-05 14:46:05 +00:00
dependabot[bot] 1f0ba745e9 chore: bump github.com/bramvdbogaerde/go-scp (#12015)
Bumps [github.com/bramvdbogaerde/go-scp](https://github.com/bramvdbogaerde/go-scp) from 1.2.1-0.20221219230748-977ee74ac37b to 1.3.0.
- [Release notes](https://github.com/bramvdbogaerde/go-scp/releases)
- [Commits](https://github.com/bramvdbogaerde/go-scp/commits/v1.3.0)

---
updated-dependencies:
- dependency-name: github.com/bramvdbogaerde/go-scp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-05 14:50:02 +02:00
dependabot[bot] c1e01dfb7b chore: bump github.com/elastic/go-sysinfo from 1.11.0 to 1.12.0 (#12013)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-05 15:37:57 +03:00
Spike Curtis e5ba586e30 fix: fix graceful disconnect in DialWorkspaceAgent (#11993)
I noticed in testing that the CLI wasn't correctly sending the disconnect message when it shuts down, and thus agents are seeing this as a "lost" peer, rather than a "disconnected" one. 

What was happening is that we just used a single context for everything from the netconn to the RPCs, and when the context was canceled we failed to send the disconnect message due to canceled context.

So, this PR splits things into two contexts, with a graceful one set to last up to 1 second longer than the main one.
2024-02-05 14:01:37 +04:00
Spike Curtis bb99cb7d2b chore: move FakeCoordinator to tailnettest (#11992)
Moves FakeCoordinator to tailnettest since it's reused in testing multiple packages in this stack of PRs.
2024-02-05 13:49:32 +04:00
Spike Curtis 646ac942b2 chore: rename FakeCoordinator for export (#11991)
Part of a stack that fixes graceful disconnect from the CLI to tailnet.  I reuse FakeCoordinator in a test for graceful disconnects.
2024-02-05 13:33:31 +04:00
Eric Paulsen f57ce97b5a docs: add faq for gateway reconnects (#12007)
* docs: add faq for gateway reconnects

* make: fmt
2024-02-04 15:50:53 -06:00
Kayla Washburn-Love 1d14d4e58c fix: use dark background in terminal, even when a light theme is selected (#12004) 2024-02-02 15:05:52 -07:00
Jon Ayers 73c5993bea fix: only display xray results if vulns > 0 (#11989) 2024-02-02 11:02:46 -06:00
Cian Johnston 6593de3c73 fix(dogfood/flake.nix): add google-chrome (#11974) 2024-02-02 15:56:06 +00:00
Bruno Quaresma 9b930f8fad feat(site): show deprecation message on template page (#11996) 2024-02-02 14:13:35 +00:00
Bruno Quaresma 2e378b4894 fix(site): fix parameter input icon shrink (#11995) 2024-02-02 13:49:49 +00:00
Mathias Fredriksson aae228ac01 fix(dogfood): resolve module.git-clone.repo_dir containing ~/ (#11994) 2024-02-02 14:21:34 +02:00
Mathias Fredriksson bddea7bcf9 feat(cli/vscodessh): add support for --wait and scripts that block login (#10473) 2024-02-02 13:18:26 +02:00
Kayla Washburn-Love c6c71de353 fix: change build status colors (#11985) 2024-02-01 18:02:40 -07:00
dependabot[bot] efac9ced3e chore: bump github.com/moby/moby (#11975)
Bumps [github.com/moby/moby](https://github.com/moby/moby) from 24.0.1+incompatible to 25.0.2+incompatible.
- [Release notes](https://github.com/moby/moby/releases)
- [Commits](https://github.com/moby/moby/compare/v24.0.1...v25.0.2)

---
updated-dependencies:
- dependency-name: github.com/moby/moby
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-01 13:58:13 -06:00
Muhammad Atif Ali 21237d96a5 docs: update remote docker host docs (#11919)
* docs: update remote docker host docs

Adds a link to external provisioners as a method to use remote docker hosts

* `make fmt`

* Update docker.md

* fmt
2024-02-01 22:43:38 +03:00
Muhammad Atif Ali 9616b92f0e chore(dogfood): fix nix icon path (#11984) 2024-02-01 19:27:41 +00:00
Bruno Quaresma 96346525e0 fix(site): fix text overflow on batch ws deletion (#11981)
Before:
![image](https://github.com/coder/coder/assets/3165839/723a8fd7-8f63-4712-8af1-cd442455c723)

After:
<img width="674" alt="Screenshot 2024-02-01 at 13 48 56" src="https://github.com/coder/coder/assets/3165839/91c3099e-6a11-4beb-b46b-70a9a6c4abb4">
2024-02-01 14:02:08 -03:00
Marcin Tojek ad8e0db172 feat: add custom error message on signups disabled page (#11959) 2024-02-01 18:01:25 +01:00
Kayla Washburn-Love e070a55142 refactor: stabilize theme.roles (#11969) 2024-02-01 09:53:26 -07:00
Bruno Quaresma 6c9f60a9c5 refactor(site): only display quota if it is higher than 0 (#11979) 2024-02-01 13:49:48 -03:00
Steven Masley 79d5c238cc fix: always return a clean http client for promoauth (#11963)
* fix: add unit test to verify default client is not broken

* always return a clean http client
* No need to clone the tripper
2024-02-01 11:13:34 -05:00
Bruno Quaresma 1a94686928 refactor(site): add table chosmetic changes (#11977)
- Set default 14px as the default font size for the table content
- Add `xsmall` size for checkboxes
- Remove checkbox wrapper padding on the table heading

Before:
<img width="1512" alt="Screenshot 2024-02-01 at 10 22 10" src="https://github.com/coder/coder/assets/3165839/92b844ae-f2bf-476a-89fe-90b16f19c306">

After: 
<img width="1512" alt="Screenshot 2024-02-01 at 10 26 00" src="https://github.com/coder/coder/assets/3165839/0f87d098-4b13-4373-96d2-2c18ee2587f6">
2024-02-01 10:41:15 -03:00
Spike Curtis 1aa117b9ec chore: rename client Listen to ConnectRPC (#11916)
ConnectRPC seems more appropriate for this function
2024-02-01 14:44:11 +04:00
dependabot[bot] 1031ccb3c9 chore: bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 (#11968)
Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.5 to 1.1.12.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.12/CHANGELOG.md)
- [Commits](https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.12)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-01 13:15:01 +03:00
Spike Curtis d5a98cc6d7 fix: avoid race in TestPGPubsub_Metrics by using Eventually (#11973)
Annoyingly, prometheus Registry collects metrics async, which is causing our test to be racy.  They also don't export enough from the Metric interface for us to replicate a synchronous collect, so we have to use Eventually to test.
2024-02-01 12:10:19 +04:00
Spike Curtis 5a359d50dd feat: add metrics to PGPubsub (#11971)
Adds prometheus metrics to PGPubsub for monitoring its health and performance in production.

Related to #11950 --- additional diagnostics to help figure out what's happening
2024-02-01 11:25:03 +04:00
Asher e748312193 fix(dogfood): fix startup script looping (#11972)
Seems to be on account of the quotes interpreting a ~ literally.  We do
replace it with /home/coder but only if it matches ~/, not ~ alone.
2024-01-31 21:33:02 -09:00
Colin Adler 3ace7982aa fix: rewrite url to agent ip in single tailnet (#11810)
This restores previous behavior of being able to cache connections
across agents in single tailnet.
2024-02-01 00:25:52 -06:00
Spike Curtis 073d1f7078 chore: remove pingWebSocket since yamux runs keepalives (#11914)
Since we run yamux over the websocket, we don't need to ping at the websocket layer because yamux has a 30 second keepalive mechanism enabled in the default config.
2024-02-01 09:48:58 +04:00
Colin Adler 4ed1f5581a chore(coderd): add logging to agent rpc yamux conn (#11965) 2024-01-31 23:17:20 -06:00
Spike Curtis cc0dc103b6 chore: remove agentsdk client RPC() function (#11913)
The RPC() function isn't called, since Listen() was modified to do this job.

Listen() has the right signature, since it returns a drpc.Conn, rather than the Agent API.  That's because tailnet v2 and agent v2 are separate APIs served over the same connection.

It might be clearer to rename `Listen()` to `RPC()` but I'll save that for a different PR.
2024-02-01 08:22:12 +04:00
Spike Curtis eb03e4490a feat: add statsReporter for reporting stats on agent v2 API (#11920)
Adds a new statsReporter subcomponent of the agent, which in a later PR will be used to report stats over the v2 API.

Refactors the logic a bit so that we can handle starting and stopping stats reporting if the agent API connection drops and reconnects.
2024-02-01 08:21:01 +04:00
Spike Curtis b79785c86f feat: move agent v2 API connection monitoring to yamux layer (#11910)
Moves monitoring of the agent v2 API connection to the yamux layer.

Present behavior monitors this at the websocket layer, and closes the websocket on completion. This can cause yamux to hit unexpected errors since the connection is closed underneath it.

This might be the cause of yamux errors that some customers are seeing

![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/tCz4CxRU9jhAJ7zH8RTi/53b8b5ef-e9e5-44a5-b559-99c37c136071.png)

In any case, it's more graceful to close yamux first and let yamux close the underlying websocket.  That should limit yamux error logging to truly unexpected/error cases.

The only downside is that the yamux `Close()` doesn't accept a reason, so if the agent becomes outdated and we close the API connection, the agent just sees the connection close without a reason.  I'm not sure we log this at the agent anyway, but it would be nice.  I think more accurate logging on Coderd are more important.

I've also added some logging when the monitor disconnects for reasons other than the context being canceled (e.g. agent outdated, failed pings).
2024-02-01 08:18:35 +04:00
Spike Curtis 13e214f7f1 feat: add logging to agent yamux session (#11912)
Log yamux errors and warnings in the agent.
2024-02-01 08:18:13 +04:00
Michael Smith b0a855caa4 fix: improve click UX and styling for Auth Token page (#11863)
* wip: commit progress for clipboard update

* wip: push more progress

* chore: finish initial version of useClipboard revamp

* refactor: update API query to use newer RQ patterns

* fix: update importers of useClipboard

* fix: increase clickable area of CodeExample

* fix: update styles for CliAuthPageView

* fix: resolve issue with ref re-routing

* docs: update comments for clarity

* wip: commit progress on clipboard tests

* chore: add extra test case for referential stability

* wip: disable test stub to avoid breaking CI

* wip: add test case for tab-switching

* feat: finish changes

* fix: improve styling for strong text

* fix: make sure period doesn't break onto separate line

* fix: make center styling more friendly to screen readers

* refactor: clean up mocking implementation

* fix: resolve security concern for clipboard text

* fix: update CodeExample to obscure text when appropriate

* fix: apply secret changes to relevant code examples

* refactor: simplify code for obfuscating text

* fix: partially revert clipboard changes

* fix: clean up page styling further

* fix: remove duplicate property identifier

* refactor: rename variables for clarity

* fix: simplify/revert CopyButton component design

* fix: update how dummy input is hidden from page

* fix: remove unused onClick handler prop

* fix: resolve unused import

* fix: opt code examples out of secret behavior
2024-01-31 21:25:30 -05:00
Colin Adler c7f51a9d70 chore(site): update time until shutdown tooltip language (#11964) 2024-01-31 16:21:35 -06:00
Kayla Washburn-Love d2e6405322 chore: add inactive role to experimental theme (#11967) 2024-01-31 15:16:17 -07:00
Bruno Quaresma 4df913372f feat(site): display xray scan result in the agent (#11955) 2024-01-31 19:16:01 -03:00
Steven Masley ac64155282 fix: strip timezone information from a date in dau response (#11962)
* fix: strip timezone information from a date in dau response

Timezone information is lost, so do not forward it to the client.

* fix: timezone offset should be flipped
* Make tests deterministic
2024-01-31 16:01:50 -06:00
Kayla Washburn-Love 76e73287a5 refactor: add modules/templates and modules/workspaces (#11947) 2024-01-31 12:09:36 -07:00
Muhammad Atif Ali 4604db072a fix(dogfood): fix startup script on workspace creation (#11958) 2024-01-31 18:01:22 +03:00
Muhammad Atif Ali d2b4d58e96 chore(dogfood): use better names for image options (#11957) 2024-01-31 14:36:01 +00:00
Muhammad Atif Ali 215a9d1b30 chore: experiment building dogfood image with nix (#11680) 2024-01-31 14:27:11 +00:00
Marcin Tojek 13cbca679e feat: support template bundles as zip archives (#11839) 2024-01-31 14:49:55 +01:00
Mathias Fredriksson b25deaae20 fix(coderd/database): fix limit in GetUserWorkspaceBuildParameters (#11954) 2024-01-31 13:56:36 +02:00
Spike Curtis a34cada09a feat: add logging to pgPubsub (#11953)
Should be helpful for #11950

Adds a logger to pgPubsub and logs various events, most especially connection and disconnection from postgres.
2024-01-31 15:49:16 +04:00
Spike Curtis 1c8b803785 feat: add logging to pgcoord subscribe/unsubscribe (#11952)
Adds logging to unsubscribing from peer and tunnel updates in pgcoordinator, since #11950 seems to be problem with these subscriptions
2024-01-31 12:15:58 +04:00
Jon Ayers 0c30dde9b5 feat: add customizable upgrade message on client/server version mismatch (#11587) 2024-01-30 17:11:37 -06:00
Ammar Bandukwala adbb025e74 feat: add user-level parameter autofill (#11731)
This PR solves #10478 by auto-filling previously used template values in create and update workspace flows.

I decided against explicit user values in settings for these reasons:

* Autofill is far easier to implement
* Users benefit from autofill _by default_ — we don't need to teach them new concepts
* If we decide that autofill creates more harm than good, we can remove it without breaking compatibility
2024-01-30 16:02:21 -06:00
Kayla Washburn-Love aeb4112513 chore: update storybook (#11936) 2024-01-30 14:23:40 -07:00
Spike Curtis 520b12e1a2 fix: close MultiAgentConn when coordinator closes (#11941)
Fixes an issue where a MultiAgentConn isn't closed properly when the coordinator it is connected to is closed.

Since servertailnet checks whether the conn is closed before reinitializing, it is important that we check this, otherwise servertailnet can get stuck if the coordinator closes (e.g. when we switch from AGPL to PGCoordinator after decoding a license).
2024-01-31 00:38:19 +04:00
Colin Adler 2fd1a726aa fix: only delete expired agents on success (#11940) 2024-01-30 14:11:45 -06:00
Colin Adler 27f3b7a814 fix: add timeout to listening ports request (#11935)
This can potentially hang for 15m if the agent is unreachable.
2024-01-30 13:53:52 -06:00
Bruno Quaresma 7f1c808ff9 feat(site): simplify create template form by removing advanced settings (#11918) 2024-01-30 16:40:59 -03:00
Kayla Washburn-Love 619bdd1e7a refactor: redesign Paywall component (#11907) 2024-01-30 10:26:19 -07:00
Kayla Washburn-Love 20dcefa156 add an interaction test to InfoTooltip (#11905) 2024-01-30 10:20:11 -07:00
Bruno Quaresma e26ba1affd feat(site): do not show popover on update deadline (#11921) 2024-01-30 14:11:15 -03:00
Bruno Quaresma dcab6fa5a4 feat(site): display user avatar (#11893)
* add owner API to workspace and workspace build responses
* display user avatar in workspace top bar

Co-authored-by: Cian Johnston <cian@coder.com>
2024-01-30 17:07:06 +00:00
Mathias Fredriksson 83eea2d323 feat(scaletest/templates): add support for concurrent scenarios (#11753) 2024-01-30 14:54:54 +02:00
Bruno Quaresma 4b27c77969 fix(site): fix parameters' request upon template variables update (#11898)
Fix https://github.com/coder/coder/issues/11870
2024-01-30 08:03:53 -03:00
Mathias Fredriksson 60653bbacb fix(cli): allow template name length of 32 in template push and create (#11915) 2024-01-30 12:47:10 +02:00
Muhammad Atif Ali 86e33257af chore(docs): fix a typo (#11895) 2024-01-30 12:00:25 +03:00
Spike Curtis 0fc177203e feat: use agent v2 API to update app health (#11889)
Use the Agent v2 API to update App Health
2024-01-30 11:35:12 +04:00
Spike Curtis 2599850e54 feat: use agent v2 API to post startup (#11877)
Uses the v2 Agent API to post startup information.
2024-01-30 11:23:28 +04:00
Spike Curtis da8bb1c198 feat: use agent v2 API to fetch manifest (#11832)
Agent uses the v2 API to obtain the manifest, instead of the HTTP API.
2024-01-30 10:11:28 +04:00
Spike Curtis 9cf4e7f15a fix: prevent agent_test.go from failing on error logs (#11909)
We're failing tests on error logs like this: https://github.com/coder/coder/actions/runs/7706053882/job/21000984583

Unfortunately, the error we hit, when the underlying connection is closed, is unexported, so we can't specifically ignore it.

Part of the issue is that agent.Close() doesn't wait for these goroutines to complete before returning, so the test harness proceeds to close the connection. This looks to our product code like the network connection failing.  It would be possible to fix this, but just doesn't seem worth it for the extra insurance of catching other error logs in these tests.
2024-01-30 10:04:01 +04:00
Spike Curtis d3983e4dba feat: add logging to client tailnet yamux (#11908)
Adds logging to yamux when used for tailnet client connections, e.g. CLI and wsproxy.  This could be useful for debugging connection issues with tailnet v2 API.
2024-01-30 09:58:59 +04:00
Spike Curtis 0eff646c31 chore: move proto to sdk conversion to agentsdk (#11831)
`agentsdk` depends on `agent/proto` because it needs to get the version to dial.

Therefore, the conversion routines need to live in `agentsdk` so that we can convert to and from the Manifest.

I briefly considered refactoring the agent to only reference `proto.Manifest`, but decided against it because we might have multiple protocol versions in the future, its useful to have a protocol-independent data structure.
2024-01-30 09:04:56 +04:00
Spike Curtis 1e8a9c09fe chore: remove legacy wsconncache (#11816)
Fixes #8218

Removes `wsconncache` and related "is legacy?" functions and API calls that were used by it.

The only leftover is that Agents still use the legacy IP, so that back level clients or workspace proxies can dial them correctly.

We should eventually remove this: #11819
2024-01-30 07:56:36 +04:00
Spike Curtis 13e24f21e4 feat: use Agent v2 API for Service Banner (#11806)
Agent uses the v2 API for the service banner, rather than the v1 HTTP API.

One of several for #10534
2024-01-30 07:44:47 +04:00
Jon Ayers 4f5a2f0a9b feat: add backend for jfrog xray support (#11829) 2024-01-29 19:30:02 -06:00
dependabot[bot] 46d92dac57 ci: bump the github-actions group with 5 updates (#11890)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:38:59 +03:00
dependabot[bot] 5937027c86 chore: bump github.com/gohugoio/hugo from 0.121.2 to 0.122.0 (#11883)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:36:23 +03:00
dependabot[bot] 4dc6a302f2 chore: bump google.golang.org/grpc from 1.60.1 to 1.61.0 (#11885)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:36:12 +03:00
dependabot[bot] 3b65a1508c chore: bump github.com/google/uuid from 1.5.0 to 1.6.0 (#11886)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:36:01 +03:00
dependabot[bot] 71b79eace4 chore: bump alpine from 3.19.0 to 3.19.1 in /scripts (#11887)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:35:44 +03:00
Eric Paulsen d8a3ebef31 docs: fix example template README 404s and semantics (#11903)
* docs: fix example template README 404s and semantics

* make: gen
2024-01-29 21:34:12 +00:00
Kayla Washburn-Love f572e18144 fix: fix cliui prompt styling (#11899) 2024-01-29 13:56:43 -07:00
Spike Curtis 207328ca50 feat: use appearance.Fetcher in agentapi (#11770)
This PR updates the Agent API to use the appearance.Fetcher, which is set by entitlement code in Enterprise coderd.

This brings the agentapi into compliance with the Enterprise feature.
2024-01-29 21:22:50 +04:00
Garrett Delfosse f54278cdfe fix: respect wait flag on ping (#11896) 2024-01-29 11:50:35 -05:00
Colin Adler bc14e926d8 feat: add option to speedtest to dump a pcap of network traffic (#11848) 2024-01-29 09:57:31 -06:00
Spike Curtis b2bc3fff33 fix: wait for new template version before promoting (#11874)
Fixes a test flake due to not waiting for the correct template version prior to promoting it.
2024-01-29 19:29:56 +04:00
Steven Masley 04a23261e6 chore: ensure github uids are unique (#11826) 2024-01-29 09:13:46 -06:00
Steven Masley d66e6e78ee fix: always attempt external auth refresh when fetching (#11762) (#11830)
* fix: always attempt external auth refresh when fetching
* refactor validate to check expiry when considering "valid"
2024-01-29 08:55:15 -06:00
Cian Johnston eeef56a655 feat(cli): show workspace favorite status in list output (#11878) 2024-01-29 14:14:12 +00:00
Cian Johnston 9abf6ec170 feat(site): show favorite workspaces in ui (#11875)
* Add Star beside workspace name to indicate favorite status in WorkspacesList
* Add button in workspace top row to toggle workspace favorite status
2024-01-29 13:39:31 +00:00
Bruno Quaresma acd22b2c65 fix(site): fix capitalized username (#11891)
Fix #11888
2024-01-29 10:24:19 -03:00
Mathias Fredriksson 3e89ba23e5 test(scaletest): fix websocket error during close (#11879)
Fixes #11735
2024-01-29 13:42:30 +02:00
Muhammad Atif Ali 8398b4188b ci: fix winget-release workflow (#11865) 2024-01-29 13:49:20 +03:00
Spike Curtis bc4ae53261 chore: refactor Appearance to an interface callable by AGPL code (#11769)
The new Agent API needs an interface for ServiceBanners, so this PR creates it and refactors the AGPL and Enterprise code to achieve it.

Before we depended on the fact that the HTTP endpoint was missing to serve an empty ServiceBanner on AGPL deployments, but that won't work with dRPC, so we need a real interface to call.
2024-01-29 12:17:31 +04:00
Marcin Tojek aacb4a2b4c feat: use map instead of slice in metrics aggregator (#11815) 2024-01-29 09:12:41 +01:00
Spike Curtis 37e9479815 fix: fix TestServiceBanners/Agent (#11768)
The original test is bugged in that it

1. creates a new AGPL coderd with a new database, so no appearance is set in the DB.
2. overwrites the agentClient so the assertion after removing the license is against the AGPL coderd
2024-01-29 11:56:33 +04:00
Spike Curtis f9fdd44510 feat: change codersdk to use tailnet v2 for DERPMap updates (#11736)
fixes #10533


refactors `codersdk` workspace agent dialer to use a single websocket connection to the tailnet v2 API for both coordination and DERPMap updates, rather than separate websockets (and the v1 API for DERPMaps).
2024-01-29 11:26:50 +04:00
Muhammad Atif Ali 699a4b8dd4 chore(dogfood): use built-in VS Code Desktop button over the module (#11869) 2024-01-29 00:37:22 +03:00
Eric Paulsen be4d5221ba docs: add guide for azure federation (#11864)
* docs: add guide for azure federation

* make: fmt

* refactor: arm secrets and semantics
2024-01-28 15:51:11 -05:00
Muhammad Atif Ali 2f9bf1ebe1 ci: validate template before pushing (#11867) 2024-01-27 10:02:10 +03:00
Spike Curtis 4825b7ccd2 fix: use new context after t.Parallel in TestOAuthAppSecrets
c.f. https://coder.com/blog/go-testing-contexts-and-t-parallel

fixes flakes like https://github.com/coder/coder/runs/20856469613
2024-01-27 08:45:43 +04:00
Muhammad Atif Ali de6d4794dc chore(dogfood): replace repo_dir with base_repo_dir in git-clone module (#11835)
Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2024-01-27 01:45:07 +00:00
Kayla Washburn-Love bb8ce7bc02 chore: move components/Resources to modules/resources (#11852) 2024-01-26 15:11:07 -07:00
Kayla Washburn-Love 8f46beef72 chore: remove most usage of PropsWithChildren (#11859) 2024-01-26 12:04:22 -07:00
Kayla Washburn-Love c2b6e204f3 fix: type error from theme update (#11844) 2024-01-26 10:56:19 -07:00
Kayla Washburn-Love c85fc3c8fe refactor: add more fill variants to the experimental theme (#11827) 2024-01-26 10:28:59 -07:00
Cian Johnston 42e997d39e fix(coderd/rbac): do not cache context cancellation errors (#11840)
#7439 added global caching of RBAC results.
Calls are cached based on hash(subject, object, action).
We often use dbauthz.AsSystemRestricted to handle "internal" authz calls, and these are often repeated with similar arguments and are likely to get cached.
So a transient error doing an authz check on a system function will be cached for up to a minute.
I'm just starting off with excluding context.Canceled but there's likely a whole suite of different errors we want to also exclude from the global cache.
2024-01-26 16:19:55 +00:00
Spike Curtis d6baa3cab0 fix: stop running tests that exec sh scripts in parallel (#11834)
Ok, so my last attempt at a fix here failed

https://github.com/coder/coder/actions/runs/7666229961/job/20893608286

I have a new theory: it's not the `terraform` binary that's busy, it's actually `fake_cancel.sh` and it gets marked busy when we `exec` it from the script we write.

Use of `exec` also replaces the executing code in place, rather than starting a new process/shell, so that's why the error we get says `terraform` is busy.
2024-01-26 19:22:35 +04:00
Bruno Quaresma 0ba035a16d refactor(site): improve parameters field (#11802) 2024-01-26 11:31:52 -03:00
Kira Pilot 4c71cccbc3 fix(site): disable autostart and autostop according to template settings (#11809)
* fix (site): disable autostart and autostop according to template settings

* checking form values again; wrote tests

* fixed closure and label bugs

* fix broken query key

* tweaks
2024-01-26 09:06:01 -05:00
Mathias Fredriksson 52c08a98bb test(scaletest): fix worksapcebuild retry (#11836) 2024-01-26 15:49:19 +02:00
Mathias Fredriksson 02124758fb feat(cli/exp): extend scaletest create-workspaces with --retry option (#11825)
Part of #11801
2024-01-26 11:29:48 +00:00
Cian Johnston fdf9f03097 fix(enterprise/cli): add ID to default columns in licenses list output (#11823) 2024-01-26 09:55:16 +00:00
Dean Sheather 29707099d7 chore: add agentapi tests (#11269) 2024-01-26 07:04:19 +00:00
Muhammad Atif Ali 541154b74b docs: simplify JFrog integration docs (#11787) 2024-01-25 19:50:06 -05:00
Steven Masley 005c014f13 chore: instrument additional github api calls (#11824)
* chore: instrument additional githubapi calls

This only affects github as a login source, not external auth.
2024-01-25 18:34:46 -06:00
Bruno Quaresma e371716b38 refactor(site): add minor workspace improvements (#11822) 2024-01-25 21:05:29 -03:00
Kayla Washburn-Love 73a6899f2c chore: miscellaneous cleanup (#11785) 2024-01-25 14:22:52 -07:00
Ammar Bandukwala 79568bf628 Revert "fix: always attempt external auth refresh when fetching (#11762)"
This reverts commit 0befc0826a.
2024-01-25 14:22:47 -06:00
Steven Masley 0befc0826a fix: always attempt external auth refresh when fetching (#11762)
* fix: always attempt external auth refresh when fetching
* refactor validate to check expiry when considering "valid"
2024-01-25 10:54:56 -06:00
Bruno Quaresma fd7f85bc5e fix(site): fix proxy settings link (#11817) 2024-01-25 12:16:24 +00:00
Cian Johnston 8eae4f83bf fix(coderd/provisionerdserver): fix test flake in TestHeartbeat (#11808) 2024-01-25 12:05:57 +00:00
Muhammad Atif Ali 979a920832 docs: use coder modules in offline deployments (#11788)
* docs: use coder modules in offline deployments

* fix typos

* Update offline installation instructions with Artifactory support for Coder modules

* Review suggestions
2024-01-25 08:01:56 +03:00
Ben Potter 6b0e1291d2 docs: add v2.7.3 changelog (#11811)
* docs: add v2.7.1 changelog

* docs: add v2.7.2 changelog
2024-01-24 16:53:08 -06:00
Kayla Washburn-Love 3d76e1b55c chore: clean up package.json and tsconfig (#11757) 2024-01-24 13:53:44 -07:00
Cian Johnston ecae6f9135 fix(enterprise/tailnet): handle query canceled error in sendBeat() (#11794) 2024-01-24 18:42:05 +00:00
Bruno Quaresma 8bc91b489e refactor(site): increase form fields gap (#11803) 2024-01-24 14:16:42 -03:00
Marcin Tojek 560e8cc1ae fix: check update permission to start workspace (#11798) 2024-01-24 17:18:03 +01:00
Cian Johnston 4616ccf462 fix(coderd): alter return signature of convertWorkspace, add check for requesterID (#11796) 2024-01-24 14:13:14 +00:00
Cian Johnston 70dc282b7d feat(cli): add favorite/unfavorite commands (#11793) 2024-01-24 14:05:39 +00:00
Cian Johnston f92336c4d5 feat(coderd): allow workspace owners to mark workspaces as favorite (#11791)
- Adds column `favorite` to workspaces table
- Adds API endpoints to favorite/unfavorite workspaces
- Modifies sorting order to return owners' favorite workspaces first
2024-01-24 13:39:19 +00:00
Bruno Quaresma 6145da8a9e refactor(site): verify external auth before display ws form (#11777) 2024-01-24 09:45:22 -03:00
Spike Curtis 5cbb76b47a fix: stop spamming DERP map updates for equivalent maps (#11792)
Fixes 2 related issues:

1. wsconncache had incorrect logic to test whether to send DERPMap updates, sending if the maps were equivalent, instead of if they were _not equivalent_.
2. configmaps used a bugged check to test equality between DERPMaps, since it contains a map and the map entries are serialized in random order. Instead, we avoid comparing the protobufs and instead depend on the existing function that compares `tailcfg.DERPMap`. This also has the effect of reducing the number of times we convert to and from protobuf.
2024-01-24 16:27:15 +04:00
Spike Curtis f5dbc718a7 fix: accept agent RPC connection without version query parameter (#11790)
Fixes an issue where Coder v2.7.1 agents connect to /api/v2/workspaceagents/me/rpc without a version query parameter
2024-01-24 09:10:16 +04:00
Colin Adler 13beb04521 fix: disable keepalives in workspaceapps transport (#11789)
Connection caching causes requests to hit the wrong workspaces. See
comment.

Fixes https://github.com/coder/coder/issues/11767
2024-01-24 14:46:59 +10:00
Muhammad Atif Ali 1e2634d2d0 chore(dogfood): use versioning for coder modules (#11774) 2024-01-24 01:33:25 +00:00
Kayla Washburn-Love 31a6a5dc6d chore: add stories for DropdownArrow (#11764) 2024-01-23 16:02:57 -07:00
Jon Ayers 383eed93f8 fix: use correct logger for lifecycle_executor (#11763) 2024-01-23 14:33:55 -06:00
Bruno Quaresma e828daba6e refactor(site): simplify create workspace form (#11771)
This is the first PR of a series of PRs trying to simplify and improve the create workspace flow.
- Use the existent template header and remove the selected template card
- Move the owner field to the general section so we don't have "anemic" sections with single fields

Before:
<img width="1512" alt="Screenshot 2024-01-23 at 10 22 45" src="https://github.com/coder/coder/assets/3165839/6a2ba6b4-9ffb-4576-9282-7901691f45ee">

Now:
<img width="1512" alt="Screenshot 2024-01-23 at 10 22 56" src="https://github.com/coder/coder/assets/3165839/84301548-4af9-4de0-96ff-2a6363fc8cf7">
2024-01-23 15:39:23 -03:00
Steven Masley d6ba0dfecb feat: add "updated" search param to workspaces (#11714)
* feat: add "updated" search param to workspaces
* rego -> sql needs to specify which <table>.organization_id
2024-01-23 11:52:06 -06:00
Steven Masley 081fbef097 fix: code-server path based forwarding, defer to code-server (#11759)
Do not attempt to construct a path based port forward url.
Always defer to code server, as it has it's own proxy method.
2024-01-23 11:36:44 -06:00
Marcin Tojek 77a4792ecd fix(cli): ssh: auto-update workspace (#11773) 2024-01-23 18:01:44 +01:00
Bruno Quaresma 369821ea19 feat(site): generates unique workspace names by default (#11772) 2024-01-23 15:55:29 +00:00
Bruno Quaresma 910f17f4e7 refactor(site): refactor external auth component (#11758)
Recommended improvements:
- Rename component for clarity 
- Simplify interface for contextual relevance 
- Handle polling errors based on section, not every button

Before:
<img width="1511" alt="Screenshot 2024-01-22 at 15 24 26" src="https://github.com/coder/coder/assets/3165839/cfb8c0bc-f5a2-4708-bd97-fdfc46bd1eee">

Now:
<img width="1512" alt="Screenshot 2024-01-22 at 15 24 41" src="https://github.com/coder/coder/assets/3165839/5aaad448-1bb2-45ea-9250-cd374a072be2">
2024-01-23 12:26:12 -03:00
Spike Curtis 059e533544 feat: agent uses Tailnet v2 API for DERPMap updates (#11698)
Switches the Agent to use Tailnet v2 API to get DERPMap updates.

Subsequent PRs will do the same for the CLI (`codersdk`) and `wsproxy`.
2024-01-23 14:42:07 +04:00
Spike Curtis 3e0e7f8739 feat: check agent API version on connection (#11696)
fixes #10531

Adds a check for `version` on connection to the Agent API websocket endpoint.  This is primarily for future-proofing, so that up-level agents get a sensible error if they connect to a back-level Coderd.

It also refactors the location of the `CurrentVersion` variables, to be part of the `proto` packages, since the versions refer to the APIs defined therein.
2024-01-23 14:27:49 +04:00
Spike Curtis eb12fd7d92 feat: make ServerTailnet set peers lost when it reconnects to the coordinator (#11682)
Adds support to `ServerTailnet` to set all peers lost before attempting to reconnect to the coordinator. In practice, this only really affects `wsproxy` since coderd has a local connection to the coordinator that only goes down if we're shutting down or change licenses.
2024-01-23 13:17:56 +04:00
dependabot[bot] f86186eef2 chore: bump github.com/hashicorp/terraform-json from 0.20.0 to 0.21.0 (#11738)
Bumps [github.com/hashicorp/terraform-json](https://github.com/hashicorp/terraform-json) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/hashicorp/terraform-json/releases)
- [Commits](https://github.com/hashicorp/terraform-json/compare/v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/terraform-json
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-23 09:46:11 +03:00
Ben Potter 18d43405c0 chore: change SSH wording on workspace page (#11702) 2024-01-23 09:45:44 +03:00
dependabot[bot] ca38bfd2fc ci: bump the github-actions group with 2 updates (#11745)
Bumps the github-actions group with 2 updates: [crate-ci/typos](https://github.com/crate-ci/typos) and [toshimaru/auto-author-assign](https://github.com/toshimaru/auto-author-assign).


Updates `crate-ci/typos` from 1.17.1 to 1.17.2
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.17.1...v1.17.2)

Updates `toshimaru/auto-author-assign` from 2.0.1 to 2.1.0
- [Release notes](https://github.com/toshimaru/auto-author-assign/releases)
- [Changelog](https://github.com/toshimaru/auto-author-assign/blob/main/CHANGELOG.md)
- [Commits](https://github.com/toshimaru/auto-author-assign/compare/v2.0.1...v2.1.0)

---
updated-dependencies:
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: toshimaru/auto-author-assign
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-23 09:43:02 +03:00
Stephen Kirby 91a8b1b886 chore: fix broken docs links (#11760)
* fixed broken install/packages.md link

* fixed broken docs links

* fixed admin/auth link

* fixed example-guide links

* replaced mitchell tweet with nix docs

* make fmt

* replaced private image with imgur
2024-01-22 18:45:34 -06:00
Asher 3014777d2a feat: add endpoints to oauth2 provider applications (#11718)
These will show up when configuring the application along with the
client ID and everything else.  Should make it easier to configure the
application, otherwise you will have to go look up the URLs in the
docs (which are not yet written).

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
2024-01-22 13:25:25 -09:00
Steven Masley 8e0a153725 chore: implement device auth flow for fake idp (#11707)
* chore: implement device auth flow for fake idp
2024-01-22 20:46:05 +00:00
Asher 16c6cefde8 chore: pass lifetime directly into api key generate (#11715)
Rather than passing all the deployment values.  This is to make it
easier to generate API keys as part of the oauth flow.

I also added and fixed a test for when the lifetime is set and the
default and expiration are unset.

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
2024-01-22 11:42:55 -09:00
Bruno Quaresma a31d19d538 refactor(site): apply cosmetic changes and remove ExternalAuth from settings page (#11756) 2024-01-22 16:07:43 -03:00
Asher 7589df325b fix: display error when fetching OAuth2 provider apps (#11713) 2024-01-22 09:56:36 -09:00
Kayla Washburn-Love 69e963b1a2 refactor: move dashboard functionality to modules/dashboard/ (#11721) 2024-01-22 11:44:33 -07:00
Bruno Quaresma 14f114b224 chore(site): add test for sensitive value (#11755) 2024-01-22 15:03:15 -03:00
Kayla Washburn-Love f74ef142d0 refactor: reorganize auth components and hooks (#11717) 2024-01-22 10:43:32 -07:00
Bruno Quaresma f02561a599 chore(site): minor refactor to the resource metadata code (#11746) 2024-01-22 12:55:46 -03:00
Spike Curtis 5388a1b6d7 fix: use TSMP ping for reachability, not latency (#11749)
Use TSMP ping for reachability, but leave Disco ping for when we call Ping() since we often use that to determine whether we have a direct connection.

Also adds unit tests to make sure Ping() returns direct connection vs DERP correctly.
2024-01-22 17:37:15 +04:00
1655 changed files with 73663 additions and 37114 deletions
+6
View File
@@ -0,0 +1,6 @@
# Ignore all files and folders
**
# Include flake.nix and flake.lock
!flake.nix
!flake.lock
+2
View File
@@ -6,10 +6,12 @@ coderd/apidoc/swagger.json linguist-generated=true
coderd/database/dump.sql linguist-generated=true
peerbroker/proto/*.go linguist-generated=true
provisionerd/proto/*.go linguist-generated=true
provisionerd/proto/version.go linguist-generated=false
provisionersdk/proto/*.go linguist-generated=true
*.tfplan.json linguist-generated=true
*.tfstate.json linguist-generated=true
*.tfstate.dot linguist-generated=true
*.tfplan.dot linguist-generated=true
site/e2e/provisionerGenerated.ts linguist-generated=true
site/src/api/typesGenerated.ts linguist-generated=true
site/src/pages/SetupPage/countries.tsx linguist-generated=true
+1 -1
View File
@@ -4,7 +4,7 @@ description: |
inputs:
version:
description: "The Go version to use."
default: "1.21.5"
default: "1.21.9"
runs:
using: "composite"
steps:
+34
View File
@@ -0,0 +1,34 @@
app = "jnb-coder"
primary_region = "jnb"
[experimental]
entrypoint = ["/bin/sh", "-c", "CODER_DERP_SERVER_RELAY_URL=\"http://[${FLY_PRIVATE_IP}]:3000\" /opt/coder wsproxy server"]
auto_rollback = true
[build]
image = "ghcr.io/coder/coder-preview:main"
[env]
CODER_ACCESS_URL = "https://jnb.fly.dev.coder.com"
CODER_HTTP_ADDRESS = "0.0.0.0:3000"
CODER_PRIMARY_ACCESS_URL = "https://dev.coder.com"
CODER_WILDCARD_ACCESS_URL = "*--apps.jnb.fly.dev.coder.com"
CODER_VERBOSE = "true"
[http_service]
internal_port = 3000
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
# Ref: https://fly.io/docs/reference/configuration/#http_service-concurrency
[http_service.concurrency]
type = "requests"
soft_limit = 50
hard_limit = 100
[[vm]]
cpu_kind = "shared"
cpus = 2
memory_mb = 512
+6
View File
@@ -22,6 +22,12 @@ primary_region = "cdg"
auto_start_machines = true
min_machines_running = 0
# Ref: https://fly.io/docs/reference/configuration/#http_service-concurrency
[http_service.concurrency]
type = "requests"
soft_limit = 50
hard_limit = 100
[[vm]]
cpu_kind = "shared"
cpus = 2
@@ -22,6 +22,12 @@ primary_region = "gru"
auto_start_machines = true
min_machines_running = 0
# Ref: https://fly.io/docs/reference/configuration/#http_service-concurrency
[http_service.concurrency]
type = "requests"
soft_limit = 50
hard_limit = 100
[[vm]]
cpu_kind = "shared"
cpus = 2
+6
View File
@@ -22,6 +22,12 @@ primary_region = "syd"
auto_start_machines = true
min_machines_running = 0
# Ref: https://fly.io/docs/reference/configuration/#http_service-concurrency
[http_service.concurrency]
type = "requests"
soft_limit = 50
hard_limit = 100
[[vm]]
cpu_kind = "shared"
cpus = 2
+3 -4
View File
@@ -88,10 +88,9 @@ provider "kubernetes" {
data "coder_workspace" "me" {}
resource "coder_agent" "main" {
os = "linux"
arch = "amd64"
startup_script_timeout = 180
startup_script = <<-EOT
os = "linux"
arch = "amd64"
startup_script = <<-EOT
set -e
# install and start code-server
+37 -72
View File
@@ -46,7 +46,7 @@ jobs:
fetch-depth: 1
# For pull requests it's not necessary to checkout the code
- name: check changed files
uses: dorny/paths-filter@v2
uses: dorny/paths-filter@v3
id: filter
with:
filters: |
@@ -126,12 +126,13 @@ jobs:
- name: Get golangci-lint cache dir
run: |
go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.53.2
linter_ver=$(egrep -o 'GOLANGCI_LINT_VERSION=\S+' dogfood/Dockerfile | cut -d '=' -f 2)
go install github.com/golangci/golangci-lint/cmd/golangci-lint@v$linter_ver
dir=$(golangci-lint cache status | awk '/Dir/ { print $2 }')
echo "LINT_CACHE_DIR=$dir" >> $GITHUB_ENV
- name: golangci-lint cache
uses: buildjet/cache@v3
uses: buildjet/cache@v4
with:
path: |
${{ env.LINT_CACHE_DIR }}
@@ -141,7 +142,7 @@ jobs:
# Check for any typos
- name: Check for typos
uses: crate-ci/typos@v1.17.1
uses: crate-ci/typos@v1.19.0
with:
config: .github/workflows/typos.toml
@@ -154,7 +155,7 @@ jobs:
# Needed for helm chart linting
- name: Install helm
uses: azure/setup-helm@v3
uses: azure/setup-helm@v4
with:
version: v3.9.2
@@ -201,7 +202,9 @@ jobs:
popd
- name: make gen
run: "make --output-sync -j -B gen"
# no `-j` flag as `make` fails with:
# coderd/rbac/object_gen.go:1:1: syntax error: package statement must be first
run: "make --output-sync -B gen"
- name: Check for unstaged files
run: ./scripts/check_unstaged.sh
@@ -225,7 +228,7 @@ jobs:
with:
# This doesn't need caching. It's super fast anyways!
cache: false
go-version: 1.21.5
go-version: 1.21.9
- name: Install shfmt
run: go install mvdan.cc/sh/v3/cmd/shfmt@v3.7.0
@@ -305,7 +308,7 @@ jobs:
api-key: ${{ secrets.DATADOG_API_KEY }}
- name: Check code coverage
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
# This action has a tendency to error out unexpectedly, it has
# the `fail_ci_if_error` option that defaults to `false`, but
# that is no guarantee, see:
@@ -353,7 +356,7 @@ jobs:
api-key: ${{ secrets.DATADOG_API_KEY }}
- name: Check code coverage
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
# This action has a tendency to error out unexpectedly, it has
# the `fail_ci_if_error` option that defaults to `false`, but
# that is no guarantee, see:
@@ -412,7 +415,7 @@ jobs:
working-directory: site
- name: Check code coverage
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
# This action has a tendency to error out unexpectedly, it has
# the `fail_ci_if_error` option that defaults to `false`, but
# that is no guarantee, see:
@@ -469,11 +472,19 @@ jobs:
- run: pnpm playwright:install
working-directory: site
- run: pnpm playwright:test --workers 1
# Run tests that don't require an enterprise license without an enterprise license
- run: pnpm playwright:test --forbid-only --workers 1
env:
DEBUG: pw:api
working-directory: site
# Run all of the tests with an enterprise license
- run: pnpm playwright:test --forbid-only --workers 1
env:
DEBUG: pw:api
CODER_E2E_ENTERPRISE_LICENSE: ${{ secrets.CODER_E2E_ENTERPRISE_LICENSE }}
working-directory: site
- name: Upload Playwright Failed Tests
if: always() && github.actor != 'dependabot[bot]' && runner.os == 'Linux' && !github.event.pull_request.head.repo.fork
uses: actions/upload-artifact@v4
@@ -516,7 +527,8 @@ jobs:
NODE_OPTIONS: "--max_old_space_size=4096"
STORYBOOK: true
with:
buildScriptName: "storybook:build"
# Do a fast, testing build for change previews
buildScriptName: "storybook:ci"
exitOnceUploaded: true
# This will prevent CI from failing when Chromatic detects visual changes
exitZeroOnChanges: true
@@ -530,6 +542,8 @@ jobs:
# Run TurboSnap to trace file dependencies to related stories
# and tell chromatic to only take snapshots of relevent stories
onlyChanged: true
# Avoid uploading single files, because that's very slow
zip: true
# This is a separate step for mainline only that auto accepts and changes
# instead of holding CI up. Since we squash/merge, this is defensive to
@@ -547,6 +561,7 @@ jobs:
autoAcceptChanges: true
# This will prevent CI from failing when Chromatic detects visual changes
exitZeroOnChanges: true
# Do a full build with documentation for mainline builds
buildScriptName: "storybook:build"
projectToken: 695c25b6cb65
workingDir: "./site"
@@ -554,6 +569,8 @@ jobs:
# Run TurboSnap to trace file dependencies to related stories
# and tell chromatic to only take snapshots of relevent stories
onlyChanged: true
# Avoid uploading single files, because that's very slow
zip: true
offlinedocs:
name: offlinedocs
@@ -608,8 +625,10 @@ jobs:
pnpm lint
- name: Build
# no `-j` flag as `make` fails with:
# coderd/rbac/object_gen.go:1:1: syntax error: package statement must be first
run: |
make -j build/coder_docs_"$(./scripts/version.sh)".tgz
make build/coder_docs_"$(./scripts/version.sh)".tgz
required:
runs-on: ubuntu-latest
@@ -655,7 +674,7 @@ jobs:
# to main branch. We are only building this for amd64 platform. (>95% pulls
# are for amd64)
needs: changes
if: needs.changes.outputs.docs-only == 'false' && !github.event.pull_request.head.repo.fork
if: github.ref == 'refs/heads/main' && needs.changes.outputs.docs-only == 'false' && !github.event.pull_request.head.repo.fork
runs-on: ${{ github.repository_owner == 'coder' && 'buildjet-8vcpu-ubuntu-2204' || 'ubuntu-latest' }}
env:
DOCKER_CLI_EXPERIMENTAL: "enabled"
@@ -726,7 +745,7 @@ jobs:
# Define specific tags
tags=("$tag" "main" "latest")
# Create and push a multi-arch manifest for each tag
# we are adding `latest` tag and keeping `main` for backward
# compatibality
@@ -741,7 +760,7 @@ jobs:
- name: Prune old images
if: github.ref == 'refs/heads/main'
uses: vlaurin/action-ghcr-prune@v0.5.0
uses: vlaurin/action-ghcr-prune@v0.6.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
organization: coder
@@ -849,68 +868,14 @@ jobs:
flyctl deploy --image "$IMAGE" --app paris-coder --config ./.github/fly-wsproxies/paris-coder.toml --env "CODER_PROXY_SESSION_TOKEN=$TOKEN_PARIS" --yes
flyctl deploy --image "$IMAGE" --app sydney-coder --config ./.github/fly-wsproxies/sydney-coder.toml --env "CODER_PROXY_SESSION_TOKEN=$TOKEN_SYDNEY" --yes
flyctl deploy --image "$IMAGE" --app sao-paulo-coder --config ./.github/fly-wsproxies/sao-paulo-coder.toml --env "CODER_PROXY_SESSION_TOKEN=$TOKEN_SAO_PAULO" --yes
flyctl deploy --image "$IMAGE" --app jnb-coder --config ./.github/fly-wsproxies/jnb-coder.toml --env "CODER_PROXY_SESSION_TOKEN=$TOKEN_JNB" --yes
env:
FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
IMAGE: ${{ needs.build.outputs.IMAGE }}
TOKEN_PARIS: ${{ secrets.FLY_PARIS_CODER_PROXY_SESSION_TOKEN }}
TOKEN_SYDNEY: ${{ secrets.FLY_SYDNEY_CODER_PROXY_SESSION_TOKEN }}
TOKEN_SAO_PAULO: ${{ secrets.FLY_SAO_PAULO_CODER_PROXY_SESSION_TOKEN }}
deploy-legacy-proxies:
runs-on: ubuntu-latest
timeout-minutes: 30
needs: build
if: github.ref == 'refs/heads/main' && !github.event.pull_request.head.repo.fork
permissions:
contents: read
id-token: write
steps:
- name: Authenticate to Google Cloud
uses: google-github-actions/auth@v2
with:
workload_identity_provider: projects/573722524737/locations/global/workloadIdentityPools/github/providers/github
service_account: coder-ci@coder-dogfood.iam.gserviceaccount.com
- name: Set up Google Cloud SDK
uses: google-github-actions/setup-gcloud@v2
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: coder
path: ./build
- name: Install Release
run: |
set -euo pipefail
regions=(
# gcp-region-id instance-name systemd-service-name
"australia-southeast1-b coder-sydney coder-workspace-proxy"
"europe-west3-c coder-europe coder-workspace-proxy"
"southamerica-east1-b coder-brazil coder-workspace-proxy"
)
deb_pkg=$(find ./build -name "coder_*_linux_amd64.deb" -print -quit)
if [ -z "$deb_pkg" ]; then
echo "deb package $deb_pkg not found"
ls -l ./build
exit 1
fi
gcloud config set project coder-dogfood
for region in "${regions[@]}"; do
echo "::group::$region"
set -- $region
set -x
gcloud config set compute/zone "$1"
gcloud compute scp "$deb_pkg" "${2}:/tmp/coder.deb"
gcloud compute ssh "$2" -- /bin/sh -c "set -eux; sudo dpkg -i --force-confdef /tmp/coder.deb; sudo systemctl daemon-reload; sudo service '$3' restart"
set +x
echo "::endgroup::"
done
TOKEN_JNB: ${{ secrets.FLY_JNB_CODER_PROXY_SESSION_TOKEN }}
# sqlc-vet runs a postgres docker container, runs Coder migrations, and then
# runs sqlc-vet to ensure all queries are valid. This catches any mistakes
+2 -2
View File
@@ -26,7 +26,7 @@ jobs:
pull-requests: write
steps:
- name: auto-approve dependabot
uses: hmarr/auto-approve-action@v3
uses: hmarr/auto-approve-action@v4
if: github.actor == 'dependabot[bot]'
cla:
@@ -34,7 +34,7 @@ jobs:
steps:
- name: cla
if: (github.event.comment.body == 'recheck' || github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA') || github.event_name == 'pull_request_target'
uses: contributor-assistant/github-action@v2.3.1
uses: contributor-assistant/github-action@v2.3.2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# the below token should have repo scope and must be manually added by you in the repository's secret
+33 -3
View File
@@ -7,10 +7,14 @@ on:
paths:
- "dogfood/**"
- ".github/workflows/dogfood.yaml"
- "flake.lock"
- "flake.nix"
pull_request:
paths:
- "dogfood/**"
- ".github/workflows/dogfood.yaml"
- "flake.lock"
- "flake.nix"
workflow_dispatch:
jobs:
@@ -45,7 +49,7 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
- name: Build and push
- name: Build and push Non-Nix image
uses: depot/build-push-action@v1
with:
project: b4q6ltmpzh
@@ -53,33 +57,59 @@ jobs:
buildx-fallback: true
context: "{{defaultContext}}:dogfood"
pull: true
save: true
push: ${{ github.ref == 'refs/heads/main' }}
tags: "codercom/oss-dogfood:${{ steps.docker-tag-name.outputs.tag }},codercom/oss-dogfood:latest"
- name: Build and push Nix image
uses: depot/build-push-action@v1
with:
project: b4q6ltmpzh
token: ${{ secrets.DEPOT_TOKEN }}
buildx-fallback: true
context: "."
file: "dogfood/Dockerfile.nix"
pull: true
save: true
push: ${{ github.ref == 'refs/heads/main' }}
tags: "codercom/oss-dogfood-nix:${{ steps.docker-tag-name.outputs.tag }},codercom/oss-dogfood-nix:latest"
deploy_template:
needs: build_image
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Terraform
uses: ./.github/actions/setup-tf
- name: Terraform init and validate
run: |
cd dogfood
terraform init -upgrade
terraform validate
- name: Get short commit SHA
if: github.ref == 'refs/heads/main'
id: vars
run: echo "sha_short=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
- name: Get latest commit title
if: github.ref == 'refs/heads/main'
id: message
run: echo "pr_title=$(git log --format=%s -n 1 ${{ github.sha }})" >> $GITHUB_OUTPUT
- name: "Get latest Coder binary from the server"
if: github.ref == 'refs/heads/main'
run: |
curl -fsSL "https://dev.coder.com/bin/coder-linux-amd64" -o "./coder"
chmod +x "./coder"
- name: "Push template"
if: github.ref == 'refs/heads/main'
run: |
./coder templates push $CODER_TEMPLATE_NAME --directory $CODER_TEMPLATE_DIR --yes --name=$CODER_TEMPLATE_VERSION --message="$CODER_TEMPLATE_MESSAGE" --variable jfrog_url=${{ secrets.JFROG_URL }}
./coder templates push $CODER_TEMPLATE_NAME --directory $CODER_TEMPLATE_DIR --yes --name=$CODER_TEMPLATE_VERSION --message="$CODER_TEMPLATE_MESSAGE"
env:
# Consumed by Coder CLI
CODER_URL: https://dev.coder.com
+1 -1
View File
@@ -14,4 +14,4 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Assign author
uses: toshimaru/auto-author-assign@v2.0.1
uses: toshimaru/auto-author-assign@v2.1.0
+5 -5
View File
@@ -119,7 +119,7 @@ jobs:
echo "NEW=$NEW" >> $GITHUB_OUTPUT
- name: Check changed files
uses: dorny/paths-filter@v2
uses: dorny/paths-filter@v3
id: filter
with:
base: ${{ github.ref }}
@@ -163,7 +163,7 @@ jobs:
runs-on: "ubuntu-latest"
steps:
- name: Find Comment
uses: peter-evans/find-comment@v2
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ needs.get_info.outputs.PR_NUMBER }}
@@ -173,7 +173,7 @@ jobs:
- name: Comment on PR
id: comment_id
uses: peter-evans/create-or-update-comment@v3
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ needs.get_info.outputs.PR_NUMBER }}
@@ -441,7 +441,7 @@ jobs:
echo "Slack notification sent"
- name: Find Comment
uses: peter-evans/find-comment@v2
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ env.PR_NUMBER }}
@@ -450,7 +450,7 @@ jobs:
direction: last
- name: Comment on PR
uses: peter-evans/create-or-update-comment@v3
uses: peter-evans/create-or-update-comment@v4
env:
STATUS: ${{ needs.get_info.outputs.NEW == 'true' && 'Created' || 'Updated' }}
with:
+6 -1
View File
@@ -321,7 +321,7 @@ jobs:
- name: Start Packer builds
if: ${{ !inputs.dry_run }}
uses: peter-evans/repository-dispatch@v2
uses: peter-evans/repository-dispatch@v3
with:
token: ${{ secrets.CDRCI_GITHUB_TOKEN }}
repository: coder/packages
@@ -408,6 +408,11 @@ jobs:
if: ${{ !inputs.dry_run }}
steps:
- name: Sync fork
run: gh repo sync cdrci/winget-pkgs -b master
env:
GH_TOKEN: ${{ secrets.CDRCI_GITHUB_TOKEN }}
- name: Checkout
uses: actions/checkout@v4
with:
+14 -12
View File
@@ -28,14 +28,14 @@ jobs:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Go
uses: ./.github/actions/setup-go
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: go, javascript
- name: Setup Go
uses: ./.github/actions/setup-go
# Workaround to prevent CodeQL from building the dashboard.
- name: Remove Makefile
run: |
@@ -113,16 +113,8 @@ jobs:
make -j "$image_job"
echo "image=$(cat "$image_job")" >> $GITHUB_OUTPUT
- name: Run Prisma Cloud image scan
uses: PaloAltoNetworks/prisma-cloud-scan@v1
with:
pcc_console_url: ${{ secrets.PRISMA_CLOUD_URL }}
pcc_user: ${{ secrets.PRISMA_CLOUD_ACCESS_KEY }}
pcc_pass: ${{ secrets.PRISMA_CLOUD_SECRET_KEY }}
image_name: ${{ steps.build.outputs.image }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@d43c1f16c00cfd3978dde6c07f4bbcf9eb6993ca
uses: aquasecurity/trivy-action@d710430a6722f083d3b36b8339ff66b32f22ee55
with:
image-ref: ${{ steps.build.outputs.image }}
format: sarif
@@ -142,6 +134,16 @@ jobs:
path: trivy-results.sarif
retention-days: 7
# Prisma cloud scan runs last because it fails the entire job if it
# detects vulnerabilities. :|
- name: Run Prisma Cloud image scan
uses: PaloAltoNetworks/prisma-cloud-scan@v1
with:
pcc_console_url: ${{ secrets.PRISMA_CLOUD_URL }}
pcc_user: ${{ secrets.PRISMA_CLOUD_ACCESS_KEY }}
pcc_pass: ${{ secrets.PRISMA_CLOUD_SECRET_KEY }}
image_name: ${{ steps.build.outputs.image }}
- name: Send Slack notification on failure
if: ${{ failure() }}
run: |
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
file-path: "./README.md"
- name: Send Slack notification
if: failure()
if: failure() && github.event_name != 'workflow_dispatch'
run: |
curl -X POST -H 'Content-type: application/json' -d '{"msg":"Broken links found in the documentation. Please check the logs at ${{ env.LOGS_URL }}"}' ${{ secrets.DOCS_LINK_SLACK_WEBHOOK }}
echo "Sent Slack notification"
+2 -1
View File
@@ -114,6 +114,7 @@
"Signup",
"slogtest",
"sourcemapped",
"spinbutton",
"Srcs",
"stdbuf",
"stretchr",
@@ -170,10 +171,10 @@
"workspaceapps",
"workspacebuilds",
"workspacename",
"wsconncache",
"wsjson",
"xerrors",
"xlarge",
"xsmall",
"yamux"
],
"cSpell.ignorePaths": ["site/package.json", ".vscode/settings.json"],
+44 -14
View File
@@ -361,6 +361,8 @@ $(foreach chart,$(charts),build/$(chart)_helm_$(VERSION).tgz): build/%_helm_$(VE
site/out/index.html: site/package.json $(shell find ./site $(FIND_EXCLUSIONS) -type f \( -name '*.ts' -o -name '*.tsx' \))
cd site
# prevents this directory from getting to big, and causing "too much data" errors
rm -rf out/assets/
../scripts/pnpm_install.sh
pnpm build
@@ -380,32 +382,44 @@ install: build/coder_$(VERSION)_$(GOOS)_$(GOARCH)$(GOOS_BIN_EXT)
cp "$<" "$$output_file"
.PHONY: install
fmt: fmt/prettier fmt/terraform fmt/shfmt fmt/go
BOLD := $(shell tput bold)
GREEN := $(shell tput setaf 2)
RESET := $(shell tput sgr0)
fmt: fmt/eslint fmt/prettier fmt/terraform fmt/shfmt fmt/go
.PHONY: fmt
fmt/go:
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/go$(RESET)"
# VS Code users should check out
# https://github.com/mvdan/gofumpt#visual-studio-code
go run mvdan.cc/gofumpt@v0.4.0 -w -l .
.PHONY: fmt/go
fmt/eslint:
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/eslint$(RESET)"
cd site
pnpm run lint:fix
.PHONY: fmt/eslint
fmt/prettier:
echo "--- prettier"
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/prettier$(RESET)"
cd site
# Avoid writing files in CI to reduce file write activity
ifdef CI
pnpm run format:check
else
pnpm run format:write
pnpm run format
endif
.PHONY: fmt/prettier
fmt/terraform: $(wildcard *.tf)
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/terraform$(RESET)"
terraform fmt -recursive
.PHONY: fmt/terraform
fmt/shfmt: $(SHELL_SRC_FILES)
echo "--- shfmt"
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/shfmt$(RESET)"
# Only do diff check in CI, errors on diff.
ifdef CI
shfmt -d $(SHELL_SRC_FILES)
@@ -414,7 +428,7 @@ else
endif
.PHONY: fmt/shfmt
lint: lint/shellcheck lint/go lint/ts lint/helm lint/site-icons
lint: lint/shellcheck lint/go lint/ts lint/examples lint/helm lint/site-icons
.PHONY: lint
lint/site-icons:
@@ -433,6 +447,10 @@ lint/go:
golangci-lint run
.PHONY: lint/go
lint/examples:
go run ./scripts/examplegen/main.go -lint
.PHONY: lint/examples
# Use shfmt to determine the shell files, takes editorconfig into consideration.
lint/shellcheck: $(SHELL_SRC_FILES)
echo "--- shellcheck"
@@ -476,7 +494,9 @@ gen: \
site/e2e/provisionerGenerated.ts \
site/src/theme/icons.json \
examples/examples.gen.json \
tailnet/tailnettest/coordinatormock.go
tailnet/tailnettest/coordinatormock.go \
tailnet/tailnettest/coordinateemock.go \
tailnet/tailnettest/multiagentmock.go
.PHONY: gen
# Mark all generated files as fresh so make thinks they're up-to-date. This is
@@ -504,6 +524,8 @@ gen/mark-fresh:
site/src/theme/icons.json \
examples/examples.gen.json \
tailnet/tailnettest/coordinatormock.go \
tailnet/tailnettest/coordinateemock.go \
tailnet/tailnettest/multiagentmock.go \
"
for file in $$files; do
echo "$$file"
@@ -531,7 +553,7 @@ coderd/database/querier.go: coderd/database/sqlc.yaml coderd/database/dump.sql $
coderd/database/dbmock/dbmock.go: coderd/database/db.go coderd/database/querier.go
go generate ./coderd/database/dbmock/
tailnet/tailnettest/coordinatormock.go: tailnet/coordinator.go
tailnet/tailnettest/coordinatormock.go tailnet/tailnettest/multiagentmock.go tailnet/tailnettest/coordinateemock.go: tailnet/coordinator.go tailnet/multiagent.go
go generate ./tailnet/tailnettest/
tailnet/proto/tailnet.pb.go: tailnet/proto/tailnet.proto
@@ -568,7 +590,8 @@ provisionerd/proto/provisionerd.pb.go: provisionerd/proto/provisionerd.proto
site/src/api/typesGenerated.ts: $(wildcard scripts/apitypings/*) $(shell find ./codersdk $(FIND_EXCLUSIONS) -type f -name '*.go')
go run ./scripts/apitypings/ > $@
pnpm run format:write:only "$@"
./scripts/pnpm_install.sh
pnpm exec prettier --write "$@"
site/e2e/provisionerGenerated.ts: provisionerd/proto/provisionerd.pb.go provisionersdk/proto/provisioner.pb.go
cd site
@@ -577,7 +600,8 @@ site/e2e/provisionerGenerated.ts: provisionerd/proto/provisionerd.pb.go provisio
site/src/theme/icons.json: $(wildcard scripts/gensite/*) $(wildcard site/static/icon/*)
go run ./scripts/gensite/ -icons "$@"
pnpm run format:write:only "$@"
./scripts/pnpm_install.sh
pnpm exec prettier --write "$@"
examples/examples.gen.json: scripts/examplegen/main.go examples/examples.go $(shell find ./examples/templates)
go run ./scripts/examplegen/main.go > examples/examples.gen.json
@@ -587,19 +611,23 @@ coderd/rbac/object_gen.go: scripts/rbacgen/main.go coderd/rbac/object.go
docs/admin/prometheus.md: scripts/metricsdocgen/main.go scripts/metricsdocgen/metrics
go run scripts/metricsdocgen/main.go
pnpm run format:write:only ./docs/admin/prometheus.md
./scripts/pnpm_install.sh
pnpm exec prettier --write ./docs/admin/prometheus.md
docs/cli.md: scripts/clidocgen/main.go examples/examples.gen.json $(GO_SRC_FILES)
CI=true BASE_PATH="." go run ./scripts/clidocgen
pnpm run format:write:only ./docs/cli.md ./docs/cli/*.md ./docs/manifest.json
./scripts/pnpm_install.sh
pnpm exec prettier --write ./docs/cli.md ./docs/cli/*.md ./docs/manifest.json
docs/admin/audit-logs.md: coderd/database/querier.go scripts/auditdocgen/main.go enterprise/audit/table.go coderd/rbac/object_gen.go
go run scripts/auditdocgen/main.go
pnpm run format:write:only ./docs/admin/audit-logs.md
./scripts/pnpm_install.sh
pnpm exec prettier --write ./docs/admin/audit-logs.md
coderd/apidoc/swagger.json: $(shell find ./scripts/apidocgen $(FIND_EXCLUSIONS) -type f) $(wildcard coderd/*.go) $(wildcard enterprise/coderd/*.go) $(wildcard codersdk/*.go) $(wildcard enterprise/wsproxy/wsproxysdk/*.go) $(DB_GEN_FILES) .swaggo docs/manifest.json coderd/rbac/object_gen.go
./scripts/apidocgen/generate.sh
pnpm run format:write:only ./docs/api ./docs/manifest.json ./coderd/apidoc/swagger.json
./scripts/pnpm_install.sh
pnpm exec prettier --write ./docs/api ./docs/manifest.json ./coderd/apidoc/swagger.json
update-golden-files: \
cli/testdata/.gen-golden \
@@ -614,7 +642,7 @@ update-golden-files: \
.PHONY: update-golden-files
cli/testdata/.gen-golden: $(wildcard cli/testdata/*.golden) $(wildcard cli/*.tpl) $(GO_SRC_FILES) $(wildcard cli/*_test.go)
go test ./cli -run="Test(CommandHelp|ServerYAML)" -update
go test ./cli -run="Test(CommandHelp|ServerYAML|ErrorExamples)" -update
touch "$@"
enterprise/cli/testdata/.gen-golden: $(wildcard enterprise/cli/testdata/*.golden) $(wildcard cli/*.tpl) $(GO_SRC_FILES) $(wildcard enterprise/cli/*_test.go)
@@ -755,6 +783,7 @@ test-postgres: test-postgres-docker
-count=1
.PHONY: test-postgres
# NOTE: we set --memory to the same size as a GitHub runner.
test-postgres-docker:
docker rm -f test-postgres-docker || true
docker run \
@@ -767,6 +796,7 @@ test-postgres-docker:
--name test-postgres-docker \
--restart no \
--detach \
--memory 16GB \
gcr.io/coder-dev-1/postgres:13 \
-c shared_buffers=1GB \
-c work_mem=1GB \
+11 -18
View File
@@ -23,7 +23,6 @@
[Quickstart](#quickstart) | [Docs](https://coder.com/docs) | [Why Coder](https://coder.com/why) | [Enterprise](https://coder.com/docs/v2/latest/enterprise)
[![discord](https://img.shields.io/discord/747933592273027093?label=discord)](https://discord.gg/coder)
[![codecov](https://codecov.io/gh/coder/coder/branch/main/graph/badge.svg?token=TNLW3OAP6G)](https://codecov.io/gh/coder/coder)
[![release](https://img.shields.io/github/v/release/coder/coder)](https://github.com/coder/coder/releases/latest)
[![godoc](https://pkg.go.dev/badge/github.com/coder/coder.svg)](https://pkg.go.dev/github.com/coder/coder)
[![Go Report Card](https://goreportcard.com/badge/github.com/coder/coder)](https://goreportcard.com/report/github.com/coder/coder)
@@ -53,8 +52,8 @@ curl -L https://coder.com/install.sh | sh
# Start the Coder server (caches data in ~/.cache/coder)
coder server
# Navigate to http://localhost:3000 to create your initial user
# Create a Docker template, and provision a workspace
# Navigate to http://localhost:3000 to create your initial user,
# create a Docker template, and provision a workspace
```
## Install
@@ -68,11 +67,11 @@ Releases.
curl -L https://coder.com/install.sh | sh
```
You can run the install script with `--dry-run` to see the commands that will be used to install without executing them. You can modify the installation process by including flags. Run the install script with `--help` for reference.
You can run the install script with `--dry-run` to see the commands that will be used to install without executing them. Run the install script with `--help` for additional flags.
> See [install](https://coder.com/docs/v2/latest/install) for additional methods.
Once installed, you can start a production deployment<sup>1</sup> with a single command:
Once installed, you can start a production deployment with a single command:
```shell
# Automatically sets up an external access URL on *.try.coder.app
@@ -82,8 +81,6 @@ coder server
coder server --postgres-url <url> --access-url <url>
```
> <sup>1</sup> For production deployments, set up an external PostgreSQL instance for reliability.
Use `coder --help` to get a list of flags and environment variables. Use our [install guides](https://coder.com/docs/v2/latest/install) for a full walkthrough.
## Documentation
@@ -96,19 +93,13 @@ Browse our docs [here](https://coder.com/docs/v2) or visit a specific section be
- [**Administration**](https://coder.com/docs/v2/latest/admin): Learn how to operate Coder
- [**Enterprise**](https://coder.com/docs/v2/latest/enterprise): Learn about our paid features built for large teams
## Community and Support
## Support
Feel free to [open an issue](https://github.com/coder/coder/issues/new) if you have questions, run into bugs, or have a feature request.
[Join our Discord](https://discord.gg/coder) or [Slack](https://cdr.co/join-community) to provide feedback on in-progress features, and chat with the community using Coder!
[Join our Discord](https://discord.gg/coder) to provide feedback on in-progress features, and chat with the community using Coder!
## Contributing
Contributions are welcome! Read the [contributing docs](https://coder.com/docs/v2/latest/CONTRIBUTING) to get started.
Find our list of contributors [here](https://github.com/coder/coder/graphs/contributors).
## Related
## Integrations
We are always working on new integrations. Feel free to open an issue to request an integration. Contributions are welcome in any official or community repositories.
@@ -116,10 +107,12 @@ We are always working on new integrations. Feel free to open an issue to request
- [**VS Code Extension**](https://marketplace.visualstudio.com/items?itemName=coder.coder-remote): Open any Coder workspace in VS Code with a single click
- [**JetBrains Gateway Extension**](https://plugins.jetbrains.com/plugin/19620-coder): Open any Coder workspace in JetBrains Gateway with a single click
- [**Dev Container Builder**](https://github.com/coder/envbuilder): Build development environments using `devcontainer.json` on Docker, Kubernetes, and OpenShift
- [**Module Registry**](https://registry.coder.com): Extend development environments with common use-cases
- [**Kubernetes Log Stream**](https://github.com/coder/coder-logstream-kube): Stream Kubernetes Pod events to the Coder startup logs
- [**Self-Hosted VS Code Extension Marketplace**](https://github.com/coder/code-marketplace): A private extension marketplace that works in restricted or airgapped networks integrating with [code-server](https://github.com/coder/code-server).
### Community
- [**Provision Coder with Terraform**](https://github.com/ElliotG/coder-oss-tf): Provision Coder on Google GKE, Azure AKS, AWS EKS, DigitalOcean DOKS, IBMCloud K8s, OVHCloud K8s, and Scaleway K8s Kapsule with Terraform
- [**Coder GitHub Action**](https://github.com/marketplace/actions/update-coder-template): A GitHub Action that updates Coder templates
- [**Various Templates**](./examples/templates/community-templates.md): Hetzner Cloud, Docker in Docker, and other templates the community has built.
- [**Coder Template GitHub Action**](https://github.com/marketplace/actions/update-coder-template): A GitHub Action that updates Coder templates
+1030 -498
View File
File diff suppressed because it is too large Load Diff
+353 -76
View File
@@ -5,9 +5,9 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"math/rand"
"net"
"net/http"
"net/http/httptest"
@@ -46,14 +46,16 @@ import (
"cdr.dev/slog"
"cdr.dev/slog/sloggers/sloghuman"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent"
"github.com/coder/coder/v2/agent/agentproc"
"github.com/coder/coder/v2/agent/agentproc/agentproctest"
"github.com/coder/coder/v2/agent/agentssh"
"github.com/coder/coder/v2/agent/agenttest"
"github.com/coder/coder/v2/agent/proto"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
"github.com/coder/coder/v2/cryptorand"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/coder/v2/tailnet"
"github.com/coder/coder/v2/tailnet/tailnettest"
@@ -85,11 +87,11 @@ func TestAgent_Stats_SSH(t *testing.T) {
err = session.Shell()
require.NoError(t, err)
var s *agentsdk.Stats
var s *proto.Stats
require.Eventuallyf(t, func() bool {
var ok bool
s, ok = <-stats
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 && s.SessionCountSSH == 1
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 && s.SessionCountSsh == 1
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats: %+v", s,
)
@@ -111,18 +113,18 @@ func TestAgent_Stats_ReconnectingPTY(t *testing.T) {
require.NoError(t, err)
defer ptyConn.Close()
data, err := json.Marshal(codersdk.ReconnectingPTYRequest{
data, err := json.Marshal(workspacesdk.ReconnectingPTYRequest{
Data: "echo test\r\n",
})
require.NoError(t, err)
_, err = ptyConn.Write(data)
require.NoError(t, err)
var s *agentsdk.Stats
var s *proto.Stats
require.Eventuallyf(t, func() bool {
var ok bool
s, ok = <-stats
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 && s.SessionCountReconnectingPTY == 1
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 && s.SessionCountReconnectingPty == 1
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats: %+v", s,
)
@@ -177,14 +179,14 @@ func TestAgent_Stats_Magic(t *testing.T) {
require.Eventuallyf(t, func() bool {
s, ok := <-stats
t.Logf("got stats: ok=%t, ConnectionCount=%d, RxBytes=%d, TxBytes=%d, SessionCountVSCode=%d, ConnectionMedianLatencyMS=%f",
ok, s.ConnectionCount, s.RxBytes, s.TxBytes, s.SessionCountVSCode, s.ConnectionMedianLatencyMS)
ok, s.ConnectionCount, s.RxBytes, s.TxBytes, s.SessionCountVscode, s.ConnectionMedianLatencyMs)
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 &&
// Ensure that the connection didn't count as a "normal" SSH session.
// This was a special one, so it should be labeled specially in the stats!
s.SessionCountVSCode == 1 &&
s.SessionCountVscode == 1 &&
// Ensure that connection latency is being counted!
// If it isn't, it's set to -1.
s.ConnectionMedianLatencyMS >= 0
s.ConnectionMedianLatencyMs >= 0
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats",
)
@@ -243,9 +245,9 @@ func TestAgent_Stats_Magic(t *testing.T) {
require.Eventuallyf(t, func() bool {
s, ok := <-stats
t.Logf("got stats with conn open: ok=%t, ConnectionCount=%d, SessionCountJetBrains=%d",
ok, s.ConnectionCount, s.SessionCountJetBrains)
ok, s.ConnectionCount, s.SessionCountJetbrains)
return ok && s.ConnectionCount > 0 &&
s.SessionCountJetBrains == 1
s.SessionCountJetbrains == 1
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats with conn open",
)
@@ -258,9 +260,9 @@ func TestAgent_Stats_Magic(t *testing.T) {
require.Eventuallyf(t, func() bool {
s, ok := <-stats
t.Logf("got stats after disconnect %t, %d",
ok, s.SessionCountJetBrains)
ok, s.SessionCountJetbrains)
return ok &&
s.SessionCountJetBrains == 0
s.SessionCountJetbrains == 0
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats after conn closes",
)
@@ -280,6 +282,91 @@ func TestAgent_SessionExec(t *testing.T) {
require.Equal(t, "test", strings.TrimSpace(string(output)))
}
//nolint:tparallel // Sub tests need to run sequentially.
func TestAgent_Session_EnvironmentVariables(t *testing.T) {
t.Parallel()
tmpdir := t.TempDir()
// Defined by the coder script runner, hardcoded here since we don't
// have a reference to it.
scriptBinDir := filepath.Join(tmpdir, "coder-script-data", "bin")
manifest := agentsdk.Manifest{
EnvironmentVariables: map[string]string{
"MY_MANIFEST": "true",
"MY_OVERRIDE": "false",
"MY_SESSION_MANIFEST": "false",
},
}
banner := codersdk.ServiceBannerConfig{}
session := setupSSHSession(t, manifest, banner, nil, func(_ *agenttest.Client, opts *agent.Options) {
opts.ScriptDataDir = tmpdir
opts.EnvironmentVariables["MY_OVERRIDE"] = "true"
})
err := session.Setenv("MY_SESSION_MANIFEST", "true")
require.NoError(t, err)
err = session.Setenv("MY_SESSION", "true")
require.NoError(t, err)
command := "sh"
echoEnv := func(t *testing.T, w io.Writer, env string) {
if runtime.GOOS == "windows" {
_, err := fmt.Fprintf(w, "echo %%%s%%\r\n", env)
require.NoError(t, err)
} else {
_, err := fmt.Fprintf(w, "echo $%s\n", env)
require.NoError(t, err)
}
}
if runtime.GOOS == "windows" {
command = "cmd.exe"
}
stdin, err := session.StdinPipe()
require.NoError(t, err)
defer stdin.Close()
stdout, err := session.StdoutPipe()
require.NoError(t, err)
err = session.Start(command)
require.NoError(t, err)
// Context is fine here since we're not doing a parallel subtest.
ctx := testutil.Context(t, testutil.WaitLong)
go func() {
<-ctx.Done()
_ = session.Close()
}()
s := bufio.NewScanner(stdout)
//nolint:paralleltest // These tests need to run sequentially.
for k, partialV := range map[string]string{
"CODER": "true", // From the agent.
"MY_MANIFEST": "true", // From the manifest.
"MY_OVERRIDE": "true", // From the agent environment variables option, overrides manifest.
"MY_SESSION_MANIFEST": "false", // From the manifest, overrides session env.
"MY_SESSION": "true", // From the session.
"PATH": scriptBinDir + string(filepath.ListSeparator),
} {
t.Run(k, func(t *testing.T) {
echoEnv(t, stdin, k)
// Windows is unreliable, so keep scanning until we find a match.
for s.Scan() {
got := strings.TrimSpace(s.Text())
t.Logf("%s=%s", k, got)
if strings.Contains(got, partialV) {
break
}
}
if err := s.Err(); !errors.Is(err, io.EOF) {
require.NoError(t, err)
}
})
}
}
func TestAgent_GitSSH(t *testing.T) {
t.Parallel()
session := setupSSHSession(t, agentsdk.Manifest{}, codersdk.ServiceBannerConfig{}, nil)
@@ -751,7 +838,7 @@ func TestAgent_TCPRemoteForwarding(t *testing.T) {
var ll net.Listener
var err error
for {
randomPort = pickRandomPort()
randomPort = testutil.RandomPortNoListen(t)
addr := net.TCPAddrFromAddrPort(netip.AddrPortFrom(localhost, randomPort))
ll, err = sshClient.ListenTCP(addr)
if err != nil {
@@ -1346,9 +1433,10 @@ func TestAgent_Lifecycle(t *testing.T) {
RunOnStop: true,
}},
},
make(chan *agentsdk.Stats, 50),
make(chan *proto.Stats, 50),
tailnet.NewCoordinator(logger),
)
defer client.Close()
fs := afero.NewMemMapFs()
agent := agent.New(agent.Options{
@@ -1393,56 +1481,52 @@ func TestAgent_Startup(t *testing.T) {
t.Run("EmptyDirectory", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Directory: "",
}, 0)
assert.Eventually(t, func() bool {
return client.GetStartup().Version != ""
}, testutil.WaitShort, testutil.IntervalFast)
require.Equal(t, "", client.GetStartup().ExpandedDirectory)
startup := testutil.RequireRecvCtx(ctx, t, client.GetStartup())
require.Equal(t, "", startup.GetExpandedDirectory())
})
t.Run("HomeDirectory", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Directory: "~",
}, 0)
assert.Eventually(t, func() bool {
return client.GetStartup().Version != ""
}, testutil.WaitShort, testutil.IntervalFast)
startup := testutil.RequireRecvCtx(ctx, t, client.GetStartup())
homeDir, err := os.UserHomeDir()
require.NoError(t, err)
require.Equal(t, homeDir, client.GetStartup().ExpandedDirectory)
require.Equal(t, homeDir, startup.GetExpandedDirectory())
})
t.Run("NotAbsoluteDirectory", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Directory: "coder/coder",
}, 0)
assert.Eventually(t, func() bool {
return client.GetStartup().Version != ""
}, testutil.WaitShort, testutil.IntervalFast)
startup := testutil.RequireRecvCtx(ctx, t, client.GetStartup())
homeDir, err := os.UserHomeDir()
require.NoError(t, err)
require.Equal(t, filepath.Join(homeDir, "coder/coder"), client.GetStartup().ExpandedDirectory)
require.Equal(t, filepath.Join(homeDir, "coder/coder"), startup.GetExpandedDirectory())
})
t.Run("HomeEnvironmentVariable", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Directory: "$HOME",
}, 0)
assert.Eventually(t, func() bool {
return client.GetStartup().Version != ""
}, testutil.WaitShort, testutil.IntervalFast)
startup := testutil.RequireRecvCtx(ctx, t, client.GetStartup())
homeDir, err := os.UserHomeDir()
require.NoError(t, err)
require.Equal(t, homeDir, client.GetStartup().ExpandedDirectory)
require.Equal(t, homeDir, startup.GetExpandedDirectory())
})
}
@@ -1522,7 +1606,7 @@ func TestAgent_ReconnectingPTY(t *testing.T) {
require.NoError(t, tr1.ReadUntil(ctx, matchPrompt), "find prompt")
require.NoError(t, tr2.ReadUntil(ctx, matchPrompt), "find prompt")
data, err := json.Marshal(codersdk.ReconnectingPTYRequest{
data, err := json.Marshal(workspacesdk.ReconnectingPTYRequest{
Data: "echo test\r",
})
require.NoError(t, err)
@@ -1550,7 +1634,7 @@ func TestAgent_ReconnectingPTY(t *testing.T) {
require.NoError(t, tr3.ReadUntil(ctx, matchEchoOutput), "find echo output")
// Exit should cause the connection to close.
data, err = json.Marshal(codersdk.ReconnectingPTYRequest{
data, err = json.Marshal(workspacesdk.ReconnectingPTYRequest{
Data: "exit\r",
})
require.NoError(t, err)
@@ -1670,7 +1754,7 @@ func TestAgent_UpdatedDERP(t *testing.T) {
_ = coordinator.Close()
})
agentID := uuid.New()
statsCh := make(chan *agentsdk.Stats, 50)
statsCh := make(chan *proto.Stats, 50)
fs := afero.NewMemMapFs()
client := agenttest.NewClient(t,
logger.Named("agent"),
@@ -1683,6 +1767,10 @@ func TestAgent_UpdatedDERP(t *testing.T) {
statsCh,
coordinator,
)
t.Cleanup(func() {
t.Log("closing client")
client.Close()
})
uut := agent.New(agent.Options{
Client: client,
Filesystem: fs,
@@ -1690,11 +1778,12 @@ func TestAgent_UpdatedDERP(t *testing.T) {
ReconnectingPTYTimeout: time.Minute,
})
t.Cleanup(func() {
t.Log("closing agent")
_ = uut.Close()
})
// Setup a client connection.
newClientConn := func(derpMap *tailcfg.DERPMap, name string) *codersdk.WorkspaceAgentConn {
newClientConn := func(derpMap *tailcfg.DERPMap, name string) *workspacesdk.AgentConn {
conn, err := tailnet.NewConn(&tailnet.Options{
Addresses: []netip.Prefix{netip.PrefixFrom(tailnet.IP(), 128)},
DERPMap: derpMap,
@@ -1718,13 +1807,14 @@ func TestAgent_UpdatedDERP(t *testing.T) {
if err != nil {
t.Logf("error closing in-memory coordination: %s", err.Error())
}
t.Logf("closed coordination %s", name)
})
// Force DERP.
conn.SetBlockEndpoints(true)
sdkConn := codersdk.NewWorkspaceAgentConn(conn, codersdk.WorkspaceAgentConnOptions{
sdkConn := workspacesdk.NewAgentConn(conn, workspacesdk.AgentConnOptions{
AgentID: agentID,
CloseFunc: func() error { return codersdk.ErrSkipClose },
CloseFunc: func() error { return workspacesdk.ErrSkipClose },
})
t.Cleanup(func() {
t.Logf("closing sdkConn %s", name)
@@ -1753,11 +1843,9 @@ func TestAgent_UpdatedDERP(t *testing.T) {
}
// Push a new DERP map to the agent.
err := client.PushDERPMapUpdate(agentsdk.DERPMapUpdate{
DERPMap: newDerpMap,
})
err := client.PushDERPMapUpdate(newDerpMap)
require.NoError(t, err)
t.Logf("client Pushed DERPMap update")
t.Logf("pushed DERPMap update to agent")
require.Eventually(t, func() bool {
conn := uut.TailnetConn()
@@ -1815,7 +1903,7 @@ func TestAgent_Reconnect(t *testing.T) {
defer coordinator.Close()
agentID := uuid.New()
statsCh := make(chan *agentsdk.Stats, 50)
statsCh := make(chan *proto.Stats, 50)
derpMap, _ := tailnettest.RunDERPAndSTUN(t)
client := agenttest.NewClient(t,
logger,
@@ -1826,6 +1914,7 @@ func TestAgent_Reconnect(t *testing.T) {
statsCh,
coordinator,
)
defer client.Close()
initialized := atomic.Int32{}
closer := agent.New(agent.Options{
ExchangeToken: func(ctx context.Context) (string, error) {
@@ -1859,9 +1948,10 @@ func TestAgent_WriteVSCodeConfigs(t *testing.T) {
GitAuthConfigs: 1,
DERPMap: &tailcfg.DERPMap{},
},
make(chan *agentsdk.Stats, 50),
make(chan *proto.Stats, 50),
coordinator,
)
defer client.Close()
filesystem := afero.NewMemMapFs()
closer := agent.New(agent.Options{
ExchangeToken: func(ctx context.Context) (string, error) {
@@ -1885,11 +1975,21 @@ func TestAgent_WriteVSCodeConfigs(t *testing.T) {
func TestAgent_DebugServer(t *testing.T) {
t.Parallel()
logDir := t.TempDir()
logPath := filepath.Join(logDir, "coder-agent.log")
randLogStr, err := cryptorand.String(32)
require.NoError(t, err)
require.NoError(t, os.WriteFile(logPath, []byte(randLogStr), 0o600))
derpMap, _ := tailnettest.RunDERPAndSTUN(t)
//nolint:dogsled
conn, _, _, _, agnt := setupAgent(t, agentsdk.Manifest{
DERPMap: derpMap,
}, 0)
}, 0, func(c *agenttest.Client, o *agent.Options) {
o.ExchangeToken = func(context.Context) (string, error) {
return "token", nil
}
o.LogDir = logDir
})
awaitReachableCtx := testutil.Context(t, testutil.WaitLong)
ok := conn.AwaitReachable(awaitReachableCtx)
@@ -1970,6 +2070,114 @@ func TestAgent_DebugServer(t *testing.T) {
require.Contains(t, string(resBody), `invalid state "blah", must be a boolean`)
})
})
t.Run("Manifest", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitLong)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL+"/debug/manifest", nil)
require.NoError(t, err)
res, err := srv.Client().Do(req)
require.NoError(t, err)
defer res.Body.Close()
require.Equal(t, http.StatusOK, res.StatusCode)
var v agentsdk.Manifest
require.NoError(t, json.NewDecoder(res.Body).Decode(&v))
require.NotNil(t, v)
})
t.Run("Logs", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitLong)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL+"/debug/logs", nil)
require.NoError(t, err)
res, err := srv.Client().Do(req)
require.NoError(t, err)
require.Equal(t, http.StatusOK, res.StatusCode)
defer res.Body.Close()
resBody, err := io.ReadAll(res.Body)
require.NoError(t, err)
require.NotEmpty(t, string(resBody))
require.Contains(t, string(resBody), randLogStr)
})
}
func TestAgent_ScriptLogging(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("bash scripts only")
}
t.Parallel()
ctx := testutil.Context(t, testutil.WaitMedium)
derpMap, _ := tailnettest.RunDERPAndSTUN(t)
logsCh := make(chan *proto.BatchCreateLogsRequest, 100)
lsStart := uuid.UUID{0x11}
lsStop := uuid.UUID{0x22}
//nolint:dogsled
_, _, _, _, agnt := setupAgent(
t,
agentsdk.Manifest{
DERPMap: derpMap,
Scripts: []codersdk.WorkspaceAgentScript{
{
LogSourceID: lsStart,
RunOnStart: true,
Script: `#!/bin/sh
i=0
while [ $i -ne 5 ]
do
i=$(($i+1))
echo "start $i"
done
`,
},
{
LogSourceID: lsStop,
RunOnStop: true,
Script: `#!/bin/sh
i=0
while [ $i -ne 3000 ]
do
i=$(($i+1))
echo "stop $i"
done
`, // send a lot of stop logs to make sure we don't truncate shutdown logs before closing the API conn
},
},
},
0,
func(cl *agenttest.Client, _ *agent.Options) {
cl.SetLogsChannel(logsCh)
},
)
n := 1
for n <= 5 {
logs := testutil.RequireRecvCtx(ctx, t, logsCh)
require.NotNil(t, logs)
for _, l := range logs.GetLogs() {
require.Equal(t, fmt.Sprintf("start %d", n), l.GetOutput())
n++
}
}
err := agnt.Close()
require.NoError(t, err)
n = 1
for n <= 3000 {
logs := testutil.RequireRecvCtx(ctx, t, logsCh)
require.NotNil(t, logs)
for _, l := range logs.GetLogs() {
require.Equal(t, fmt.Sprintf("stop %d", n), l.GetOutput())
n++
}
t.Logf("got %d stop logs", n-1)
}
}
// setupAgentSSHClient creates an agent, dials it, and sets up an ssh.Client for it
@@ -1987,15 +2195,17 @@ func setupSSHSession(
manifest agentsdk.Manifest,
serviceBanner codersdk.ServiceBannerConfig,
prepareFS func(fs afero.Fs),
opts ...func(*agenttest.Client, *agent.Options),
) *ssh.Session {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitLong)
defer cancel()
//nolint:dogsled
conn, _, _, fs, _ := setupAgent(t, manifest, 0, func(c *agenttest.Client, _ *agent.Options) {
opts = append(opts, func(c *agenttest.Client, o *agent.Options) {
c.SetServiceBannerFunc(func() (codersdk.ServiceBannerConfig, error) {
return serviceBanner, nil
})
})
//nolint:dogsled
conn, _, _, fs, _ := setupAgent(t, manifest, 0, opts...)
if prepareFS != nil {
prepareFS(fs)
}
@@ -2013,13 +2223,17 @@ func setupSSHSession(
}
func setupAgent(t *testing.T, metadata agentsdk.Manifest, ptyTimeout time.Duration, opts ...func(*agenttest.Client, *agent.Options)) (
*codersdk.WorkspaceAgentConn,
*workspacesdk.AgentConn,
*agenttest.Client,
<-chan *agentsdk.Stats,
<-chan *proto.Stats,
afero.Fs,
agent.Agent,
) {
logger := slogtest.Make(t, nil).Leveled(slog.LevelDebug)
logger := slogtest.Make(t, &slogtest.Options{
// Agent can drop errors when shutting down, and some, like the
// fasthttplistener connection closed error, are unexported.
IgnoreErrors: true,
}).Leveled(slog.LevelDebug)
if metadata.DERPMap == nil {
metadata.DERPMap, _ = tailnettest.RunDERPAndSTUN(t)
}
@@ -2032,28 +2246,33 @@ func setupAgent(t *testing.T, metadata agentsdk.Manifest, ptyTimeout time.Durati
if metadata.WorkspaceName == "" {
metadata.WorkspaceName = "test-workspace"
}
if metadata.WorkspaceID == uuid.Nil {
metadata.WorkspaceID = uuid.New()
}
coordinator := tailnet.NewCoordinator(logger)
t.Cleanup(func() {
_ = coordinator.Close()
})
statsCh := make(chan *agentsdk.Stats, 50)
statsCh := make(chan *proto.Stats, 50)
fs := afero.NewMemMapFs()
c := agenttest.NewClient(t, logger.Named("agent"), metadata.AgentID, metadata, statsCh, coordinator)
c := agenttest.NewClient(t, logger.Named("agenttest"), metadata.AgentID, metadata, statsCh, coordinator)
t.Cleanup(c.Close)
options := agent.Options{
Client: c,
Filesystem: fs,
Logger: logger.Named("agent"),
ReconnectingPTYTimeout: ptyTimeout,
EnvironmentVariables: map[string]string{},
}
for _, opt := range opts {
opt(c, &options)
}
closer := agent.New(options)
agnt := agent.New(options)
t.Cleanup(func() {
_ = closer.Close()
_ = agnt.Close()
})
conn, err := tailnet.NewConn(&tailnet.Options{
Addresses: []netip.Prefix{netip.PrefixFrom(tailnet.IP(), 128)},
@@ -2077,7 +2296,7 @@ func setupAgent(t *testing.T, metadata agentsdk.Manifest, ptyTimeout time.Durati
t.Logf("error closing in-mem coordination: %s", err.Error())
}
})
agentConn := codersdk.NewWorkspaceAgentConn(conn, codersdk.WorkspaceAgentConnOptions{
agentConn := workspacesdk.NewAgentConn(conn, workspacesdk.AgentConnOptions{
AgentID: metadata.AgentID,
})
t.Cleanup(func() {
@@ -2090,7 +2309,7 @@ func setupAgent(t *testing.T, metadata agentsdk.Manifest, ptyTimeout time.Durati
if !agentConn.AwaitReachable(ctx) {
t.Fatal("agent not reachable")
}
return agentConn, c, statsCh, fs, closer
return agentConn, c, statsCh, fs, agnt
}
var dialTestPayload = []byte("dean-was-here123")
@@ -2310,11 +2529,11 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
logger = slog.Make(sloghuman.Sink(io.Discard))
)
requireFileWrite(t, fs, "/proc/self/oom_score_adj", "-500")
// Create some processes.
for i := 0; i < 4; i++ {
// Create a prioritized process. This process should
// have it's oom_score_adj set to -500 and its nice
// score should be untouched.
// Create a prioritized process.
var proc agentproc.Process
if i == 0 {
proc = agentproctest.GenerateProcess(t, fs,
@@ -2332,8 +2551,8 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
},
)
syscaller.EXPECT().SetPriority(proc.PID, 10).Return(nil)
syscaller.EXPECT().GetPriority(proc.PID).Return(20, nil)
syscaller.EXPECT().SetPriority(proc.PID, 10).Return(nil)
}
syscaller.EXPECT().
Kill(proc.PID, syscall.Signal(0)).
@@ -2352,6 +2571,9 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
})
actualProcs := <-modProcs
require.Len(t, actualProcs, len(expectedProcs)-1)
for _, proc := range actualProcs {
requireFileEquals(t, fs, fmt.Sprintf("/proc/%d/oom_score_adj", proc.PID), "0")
}
})
t.Run("IgnoreCustomNice", func(t *testing.T) {
@@ -2370,8 +2592,11 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
logger = slog.Make(sloghuman.Sink(io.Discard))
)
err := afero.WriteFile(fs, "/proc/self/oom_score_adj", []byte("0"), 0o644)
require.NoError(t, err)
// Create some processes.
for i := 0; i < 2; i++ {
for i := 0; i < 3; i++ {
proc := agentproctest.GenerateProcess(t, fs)
syscaller.EXPECT().
Kill(proc.PID, syscall.Signal(0)).
@@ -2399,7 +2624,59 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
})
actualProcs := <-modProcs
// We should ignore the process with a custom nice score.
require.Len(t, actualProcs, 1)
require.Len(t, actualProcs, 2)
for _, proc := range actualProcs {
_, ok := expectedProcs[proc.PID]
require.True(t, ok)
requireFileEquals(t, fs, fmt.Sprintf("/proc/%d/oom_score_adj", proc.PID), "998")
}
})
t.Run("CustomOOMScore", func(t *testing.T) {
t.Parallel()
if runtime.GOOS != "linux" {
t.Skip("Skipping non-linux environment")
}
var (
fs = afero.NewMemMapFs()
ticker = make(chan time.Time)
syscaller = agentproctest.NewMockSyscaller(gomock.NewController(t))
modProcs = make(chan []*agentproc.Process)
logger = slog.Make(sloghuman.Sink(io.Discard))
)
err := afero.WriteFile(fs, "/proc/self/oom_score_adj", []byte("0"), 0o644)
require.NoError(t, err)
// Create some processes.
for i := 0; i < 3; i++ {
proc := agentproctest.GenerateProcess(t, fs)
syscaller.EXPECT().
Kill(proc.PID, syscall.Signal(0)).
Return(nil)
syscaller.EXPECT().GetPriority(proc.PID).Return(20, nil)
syscaller.EXPECT().SetPriority(proc.PID, 10).Return(nil)
}
_, _, _, _, _ = setupAgent(t, agentsdk.Manifest{}, 0, func(c *agenttest.Client, o *agent.Options) {
o.Syscaller = syscaller
o.ModifiedProcesses = modProcs
o.EnvironmentVariables = map[string]string{
agent.EnvProcPrioMgmt: "1",
agent.EnvProcOOMScore: "-567",
}
o.Filesystem = fs
o.Logger = logger
o.ProcessManagementTick = ticker
})
actualProcs := <-modProcs
// We should ignore the process with a custom nice score.
require.Len(t, actualProcs, 3)
for _, proc := range actualProcs {
requireFileEquals(t, fs, fmt.Sprintf("/proc/%d/oom_score_adj", proc.PID), "-567")
}
})
t.Run("DisabledByDefault", func(t *testing.T) {
@@ -2491,20 +2768,6 @@ func (s *syncWriter) Write(p []byte) (int, error) {
return s.w.Write(p)
}
// pickRandomPort picks a random port number for the ephemeral range. We do this entirely randomly
// instead of opening a listener and closing it to find a port that is likely to be free, since
// sometimes the OS reallocates the port very quickly.
func pickRandomPort() uint16 {
const (
// Overlap of windows, linux in https://en.wikipedia.org/wiki/Ephemeral_port
min = 49152
max = 60999
)
n := max - min
x := rand.Intn(n) //nolint: gosec
return uint16(min + x)
}
// echoOnce accepts a single connection, reads 4 bytes and echos them back
func echoOnce(t *testing.T, ll net.Listener) {
t.Helper()
@@ -2534,3 +2797,17 @@ func requireEcho(t *testing.T, conn net.Conn) {
require.NoError(t, err)
require.Equal(t, "test", string(b))
}
func requireFileWrite(t testing.TB, fs afero.Fs, fp, data string) {
t.Helper()
err := afero.WriteFile(fs, fp, []byte(data), 0o600)
require.NoError(t, err)
}
func requireFileEquals(t testing.TB, fs afero.Fs, fp, expect string) {
t.Helper()
actual, err := afero.ReadFile(fs, fp)
require.NoError(t, err)
require.Equal(t, expect, string(actual))
}
+8 -2
View File
@@ -2,6 +2,7 @@ package agentproctest
import (
"fmt"
"strconv"
"testing"
"github.com/spf13/afero"
@@ -29,8 +30,9 @@ func GenerateProcess(t *testing.T, fs afero.Fs, muts ...func(*agentproc.Process)
cmdline := fmt.Sprintf("%s\x00%s\x00%s", arg1, arg2, arg3)
process := agentproc.Process{
CmdLine: cmdline,
PID: int32(pid),
CmdLine: cmdline,
PID: int32(pid),
OOMScoreAdj: 0,
}
for _, mut := range muts {
@@ -45,5 +47,9 @@ func GenerateProcess(t *testing.T, fs afero.Fs, muts ...func(*agentproc.Process)
err = afero.WriteFile(fs, fmt.Sprintf("%s/cmdline", process.Dir), []byte(process.CmdLine), 0o444)
require.NoError(t, err)
score := strconv.Itoa(process.OOMScoreAdj)
err = afero.WriteFile(fs, fmt.Sprintf("%s/oom_score_adj", process.Dir), []byte(score), 0o444)
require.NoError(t, err)
return process
}
+20 -3
View File
@@ -5,6 +5,7 @@ package agentproc
import (
"errors"
"os"
"path/filepath"
"strconv"
"strings"
@@ -50,10 +51,26 @@ func List(fs afero.Fs, syscaller Syscaller) ([]*Process, error) {
}
return nil, xerrors.Errorf("read cmdline: %w", err)
}
oomScore, err := afero.ReadFile(fs, filepath.Join(defaultProcDir, entry, "oom_score_adj"))
if err != nil {
if xerrors.Is(err, os.ErrPermission) {
continue
}
return nil, xerrors.Errorf("read oom_score_adj: %w", err)
}
oom, err := strconv.Atoi(strings.TrimSpace(string(oomScore)))
if err != nil {
return nil, xerrors.Errorf("convert oom score: %w", err)
}
processes = append(processes, &Process{
PID: int32(pid),
CmdLine: string(cmdline),
Dir: filepath.Join(defaultProcDir, entry),
PID: int32(pid),
CmdLine: string(cmdline),
Dir: filepath.Join(defaultProcDir, entry),
OOMScoreAdj: oom,
})
}
+4 -3
View File
@@ -14,7 +14,8 @@ type Syscaller interface {
const defaultProcDir = "/proc"
type Process struct {
Dir string
CmdLine string
PID int32
Dir string
CmdLine string
PID int32
OOMScoreAdj int
}
+53 -10
View File
@@ -13,6 +13,7 @@ import (
"sync/atomic"
"time"
"github.com/google/uuid"
"github.com/prometheus/client_golang/prometheus"
"github.com/robfig/cron/v3"
"github.com/spf13/afero"
@@ -41,13 +42,19 @@ var (
parser = cron.NewParser(cron.Second | cron.Minute | cron.Hour | cron.Dom | cron.Month | cron.DowOptional)
)
type ScriptLogger interface {
Send(ctx context.Context, log ...agentsdk.Log) error
Flush(context.Context) error
}
// Options are a set of options for the runner.
type Options struct {
LogDir string
Logger slog.Logger
SSHServer *agentssh.Server
Filesystem afero.Fs
PatchLogs func(ctx context.Context, req agentsdk.PatchLogs) error
DataDirBase string
LogDir string
Logger slog.Logger
SSHServer *agentssh.Server
Filesystem afero.Fs
GetScriptLogger func(logSourceID uuid.UUID) ScriptLogger
}
// New creates a runner for the provided scripts.
@@ -59,6 +66,7 @@ func New(opts Options) *Runner {
cronCtxCancel: cronCtxCancel,
cron: cron.New(cron.WithParser(parser)),
closed: make(chan struct{}),
dataDir: filepath.Join(opts.DataDirBase, "coder-script-data"),
scriptsExecuted: prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: "agent",
Subsystem: "scripts",
@@ -78,6 +86,7 @@ type Runner struct {
cron *cron.Cron
initialized atomic.Bool
scripts []codersdk.WorkspaceAgentScript
dataDir string
// scriptsExecuted includes all scripts executed by the workspace agent. Agents
// execute startup scripts, and scripts on a cron schedule. Both will increment
@@ -85,6 +94,17 @@ type Runner struct {
scriptsExecuted *prometheus.CounterVec
}
// DataDir returns the directory where scripts data is stored.
func (r *Runner) DataDir() string {
return r.dataDir
}
// ScriptBinDir returns the directory where scripts can store executable
// binaries.
func (r *Runner) ScriptBinDir() string {
return filepath.Join(r.dataDir, "bin")
}
func (r *Runner) RegisterMetrics(reg prometheus.Registerer) {
if reg == nil {
// If no registry, do nothing.
@@ -104,6 +124,11 @@ func (r *Runner) Init(scripts []codersdk.WorkspaceAgentScript) error {
r.scripts = scripts
r.Logger.Info(r.cronCtx, "initializing agent scripts", slog.F("script_count", len(scripts)), slog.F("log_dir", r.LogDir))
err := r.Filesystem.MkdirAll(r.ScriptBinDir(), 0o700)
if err != nil {
return xerrors.Errorf("create script bin dir: %w", err)
}
for _, script := range scripts {
if script.Cron == "" {
continue
@@ -208,7 +233,18 @@ func (r *Runner) run(ctx context.Context, script codersdk.WorkspaceAgentScript)
if !filepath.IsAbs(logPath) {
logPath = filepath.Join(r.LogDir, logPath)
}
logger := r.Logger.With(slog.F("log_path", logPath))
scriptDataDir := filepath.Join(r.DataDir(), script.LogSourceID.String())
err := r.Filesystem.MkdirAll(scriptDataDir, 0o700)
if err != nil {
return xerrors.Errorf("%s script: create script temp dir: %w", scriptDataDir, err)
}
logger := r.Logger.With(
slog.F("log_source_id", script.LogSourceID),
slog.F("log_path", logPath),
slog.F("script_data_dir", scriptDataDir),
)
logger.Info(ctx, "running agent script", slog.F("script", script.Script))
fileWriter, err := r.Filesystem.OpenFile(logPath, os.O_CREATE|os.O_RDWR, 0o600)
@@ -238,20 +274,27 @@ func (r *Runner) run(ctx context.Context, script codersdk.WorkspaceAgentScript)
cmd.WaitDelay = 10 * time.Second
cmd.Cancel = cmdCancel(cmd)
send, flushAndClose := agentsdk.LogsSender(script.LogSourceID, r.PatchLogs, logger)
// Expose env vars that can be used in the script for storing data
// and binaries. In the future, we may want to expose more env vars
// for the script to use, like CODER_SCRIPT_DATA_DIR for persistent
// storage.
cmd.Env = append(cmd.Env, "CODER_SCRIPT_DATA_DIR="+scriptDataDir)
cmd.Env = append(cmd.Env, "CODER_SCRIPT_BIN_DIR="+r.ScriptBinDir())
scriptLogger := r.GetScriptLogger(script.LogSourceID)
// If ctx is canceled here (or in a writer below), we may be
// discarding logs, but that's okay because we're shutting down
// anyway. We could consider creating a new context here if we
// want better control over flush during shutdown.
defer func() {
if err := flushAndClose(ctx); err != nil {
if err := scriptLogger.Flush(ctx); err != nil {
logger.Warn(ctx, "flush startup logs failed", slog.Error(err))
}
}()
infoW := agentsdk.LogsWriter(ctx, send, script.LogSourceID, codersdk.LogLevelInfo)
infoW := agentsdk.LogsWriter(ctx, scriptLogger.Send, script.LogSourceID, codersdk.LogLevelInfo)
defer infoW.Close()
errW := agentsdk.LogsWriter(ctx, send, script.LogSourceID, codersdk.LogLevelError)
errW := agentsdk.LogsWriter(ctx, scriptLogger.Send, script.LogSourceID, codersdk.LogLevelError)
defer errW.Close()
cmd.Stdout = io.MultiWriter(fileWriter, infoW)
cmd.Stderr = io.MultiWriter(fileWriter, errW)
+114 -22
View File
@@ -2,13 +2,16 @@ package agentscripts_test
import (
"context"
"path/filepath"
"runtime"
"testing"
"time"
"github.com/google/uuid"
"github.com/prometheus/client_golang/prometheus"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/atomic"
"go.uber.org/goleak"
"cdr.dev/slog/sloggers/slogtest"
@@ -16,6 +19,7 @@ import (
"github.com/coder/coder/v2/agent/agentssh"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/testutil"
)
func TestMain(m *testing.M) {
@@ -24,21 +28,75 @@ func TestMain(m *testing.M) {
func TestExecuteBasic(t *testing.T) {
t.Parallel()
logs := make(chan agentsdk.PatchLogs, 1)
runner := setup(t, func(ctx context.Context, req agentsdk.PatchLogs) error {
logs <- req
return nil
ctx := testutil.Context(t, testutil.WaitShort)
fLogger := newFakeScriptLogger()
runner := setup(t, func(uuid2 uuid.UUID) agentscripts.ScriptLogger {
return fLogger
})
defer runner.Close()
err := runner.Init([]codersdk.WorkspaceAgentScript{{
Script: "echo hello",
LogSourceID: uuid.New(),
Script: "echo hello",
}})
require.NoError(t, err)
require.NoError(t, runner.Execute(context.Background(), func(script codersdk.WorkspaceAgentScript) bool {
return true
}))
log := <-logs
require.Equal(t, "hello", log.Logs[0].Output)
log := testutil.RequireRecvCtx(ctx, t, fLogger.logs)
require.Equal(t, "hello", log.Output)
}
func TestEnv(t *testing.T) {
t.Parallel()
fLogger := newFakeScriptLogger()
runner := setup(t, func(uuid2 uuid.UUID) agentscripts.ScriptLogger {
return fLogger
})
defer runner.Close()
id := uuid.New()
script := "echo $CODER_SCRIPT_DATA_DIR\necho $CODER_SCRIPT_BIN_DIR\n"
if runtime.GOOS == "windows" {
script = `
cmd.exe /c echo %CODER_SCRIPT_DATA_DIR%
cmd.exe /c echo %CODER_SCRIPT_BIN_DIR%
`
}
err := runner.Init([]codersdk.WorkspaceAgentScript{{
LogSourceID: id,
Script: script,
}})
require.NoError(t, err)
ctx := testutil.Context(t, testutil.WaitLong)
done := testutil.Go(t, func() {
err := runner.Execute(ctx, func(script codersdk.WorkspaceAgentScript) bool {
return true
})
assert.NoError(t, err)
})
defer func() {
select {
case <-ctx.Done():
case <-done:
}
}()
var log []agentsdk.Log
for {
select {
case <-ctx.Done():
require.Fail(t, "timed out waiting for logs")
case l := <-fLogger.logs:
t.Logf("log: %s", l.Output)
log = append(log, l)
}
if len(log) >= 2 {
break
}
}
require.Contains(t, log[0].Output, filepath.Join(runner.DataDir(), id.String()))
require.Contains(t, log[1].Output, runner.ScriptBinDir())
}
func TestTimeout(t *testing.T) {
@@ -46,8 +104,9 @@ func TestTimeout(t *testing.T) {
runner := setup(t, nil)
defer runner.Close()
err := runner.Init([]codersdk.WorkspaceAgentScript{{
Script: "sleep infinity",
Timeout: time.Millisecond,
LogSourceID: uuid.New(),
Script: "sleep infinity",
Timeout: time.Millisecond,
}})
require.NoError(t, err)
require.ErrorIs(t, runner.Execute(context.Background(), nil), agentscripts.ErrTimeout)
@@ -62,28 +121,61 @@ func TestCronClose(t *testing.T) {
require.NoError(t, runner.Close(), "close runner")
}
func setup(t *testing.T, patchLogs func(ctx context.Context, req agentsdk.PatchLogs) error) *agentscripts.Runner {
func setup(t *testing.T, getScriptLogger func(logSourceID uuid.UUID) agentscripts.ScriptLogger) *agentscripts.Runner {
t.Helper()
if patchLogs == nil {
if getScriptLogger == nil {
// noop
patchLogs = func(ctx context.Context, req agentsdk.PatchLogs) error {
return nil
getScriptLogger = func(uuid uuid.UUID) agentscripts.ScriptLogger {
return noopScriptLogger{}
}
}
fs := afero.NewMemMapFs()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(context.Background(), logger, prometheus.NewRegistry(), fs, 0, "")
s, err := agentssh.NewServer(context.Background(), logger, prometheus.NewRegistry(), fs, nil)
require.NoError(t, err)
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
t.Cleanup(func() {
_ = s.Close()
})
return agentscripts.New(agentscripts.Options{
LogDir: t.TempDir(),
Logger: logger,
SSHServer: s,
Filesystem: fs,
PatchLogs: patchLogs,
LogDir: t.TempDir(),
DataDirBase: t.TempDir(),
Logger: logger,
SSHServer: s,
Filesystem: fs,
GetScriptLogger: getScriptLogger,
})
}
type noopScriptLogger struct{}
func (noopScriptLogger) Send(context.Context, ...agentsdk.Log) error {
return nil
}
func (noopScriptLogger) Flush(context.Context) error {
return nil
}
type fakeScriptLogger struct {
logs chan agentsdk.Log
}
func (f *fakeScriptLogger) Send(ctx context.Context, logs ...agentsdk.Log) error {
for _, log := range logs {
select {
case <-ctx.Done():
return ctx.Err()
case f.logs <- log:
// OK!
}
}
return nil
}
func (*fakeScriptLogger) Flush(context.Context) error {
return nil
}
func newFakeScriptLogger() *fakeScriptLogger {
return &fakeScriptLogger{make(chan agentsdk.Log, 100)}
}
+73 -75
View File
@@ -32,7 +32,6 @@ import (
"github.com/coder/coder/v2/agent/usershell"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/pty"
)
@@ -55,6 +54,28 @@ const (
MagicProcessCmdlineJetBrains = "idea.vendor.name=JetBrains"
)
// Config sets configuration parameters for the agent SSH server.
type Config struct {
// MaxTimeout sets the absolute connection timeout, none if empty. If set to
// 3 seconds or more, keep alive will be used instead.
MaxTimeout time.Duration
// MOTDFile returns the path to the message of the day file. If set, the
// file will be displayed to the user upon login.
MOTDFile func() string
// ServiceBanner returns the configuration for the Coder service banner.
ServiceBanner func() *codersdk.ServiceBannerConfig
// UpdateEnv updates the environment variables for the command to be
// executed. It can be used to add, modify or replace environment variables.
UpdateEnv func(current []string) (updated []string, err error)
// WorkingDirectory sets the working directory for commands and defines
// where users will land when they connect via SSH. Default is the home
// directory of the user.
WorkingDirectory func() string
// X11SocketDir is the directory where X11 sockets are created. Default is
// /tmp/.X11-unix.
X11SocketDir string
}
type Server struct {
mu sync.RWMutex // Protects following.
fs afero.Fs
@@ -66,14 +87,10 @@ type Server struct {
// a lock on mu but protected by closing.
wg sync.WaitGroup
logger slog.Logger
srv *ssh.Server
x11SocketDir string
logger slog.Logger
srv *ssh.Server
Env map[string]string
AgentToken func() string
Manifest *atomic.Pointer[agentsdk.Manifest]
ServiceBanner *atomic.Pointer[codersdk.ServiceBannerConfig]
config *Config
connCountVSCode atomic.Int64
connCountJetBrains atomic.Int64
@@ -82,7 +99,7 @@ type Server struct {
metrics *sshServerMetrics
}
func NewServer(ctx context.Context, logger slog.Logger, prometheusRegistry *prometheus.Registry, fs afero.Fs, maxTimeout time.Duration, x11SocketDir string) (*Server, error) {
func NewServer(ctx context.Context, logger slog.Logger, prometheusRegistry *prometheus.Registry, fs afero.Fs, config *Config) (*Server, error) {
// Clients' should ignore the host key when connecting.
// The agent needs to authenticate with coderd to SSH,
// so SSH authentication doesn't improve security.
@@ -94,8 +111,29 @@ func NewServer(ctx context.Context, logger slog.Logger, prometheusRegistry *prom
if err != nil {
return nil, err
}
if x11SocketDir == "" {
x11SocketDir = filepath.Join(os.TempDir(), ".X11-unix")
if config == nil {
config = &Config{}
}
if config.X11SocketDir == "" {
config.X11SocketDir = filepath.Join(os.TempDir(), ".X11-unix")
}
if config.UpdateEnv == nil {
config.UpdateEnv = func(current []string) ([]string, error) { return current, nil }
}
if config.MOTDFile == nil {
config.MOTDFile = func() string { return "" }
}
if config.ServiceBanner == nil {
config.ServiceBanner = func() *codersdk.ServiceBannerConfig { return &codersdk.ServiceBannerConfig{} }
}
if config.WorkingDirectory == nil {
config.WorkingDirectory = func() string {
home, err := userHomeDir()
if err != nil {
return ""
}
return home
}
}
forwardHandler := &ssh.ForwardedTCPHandler{}
@@ -103,12 +141,13 @@ func NewServer(ctx context.Context, logger slog.Logger, prometheusRegistry *prom
metrics := newSSHServerMetrics(prometheusRegistry)
s := &Server{
listeners: make(map[net.Listener]struct{}),
fs: fs,
conns: make(map[net.Conn]struct{}),
sessions: make(map[ssh.Session]struct{}),
logger: logger,
x11SocketDir: x11SocketDir,
listeners: make(map[net.Listener]struct{}),
fs: fs,
conns: make(map[net.Conn]struct{}),
sessions: make(map[ssh.Session]struct{}),
logger: logger,
config: config,
metrics: metrics,
}
@@ -172,14 +211,16 @@ func NewServer(ctx context.Context, logger slog.Logger, prometheusRegistry *prom
},
}
// The MaxTimeout functionality has been substituted with the introduction of the KeepAlive feature.
// In cases where very short timeouts are set, the SSH server will automatically switch to the connection timeout for both read and write operations.
if maxTimeout >= 3*time.Second {
// The MaxTimeout functionality has been substituted with the introduction
// of the KeepAlive feature. In cases where very short timeouts are set, the
// SSH server will automatically switch to the connection timeout for both
// read and write operations.
if config.MaxTimeout >= 3*time.Second {
srv.ClientAliveCountMax = 3
srv.ClientAliveInterval = maxTimeout / time.Duration(srv.ClientAliveCountMax)
srv.ClientAliveInterval = config.MaxTimeout / time.Duration(srv.ClientAliveCountMax)
srv.MaxTimeout = 0
} else {
srv.MaxTimeout = maxTimeout
srv.MaxTimeout = config.MaxTimeout
}
s.srv = srv
@@ -400,7 +441,7 @@ func (s *Server) startPTYSession(logger slog.Logger, session ptySession, magicTy
session.DisablePTYEmulation()
if isLoginShell(session.RawCommand()) {
serviceBanner := s.ServiceBanner.Load()
serviceBanner := s.config.ServiceBanner()
if serviceBanner != nil {
err := showServiceBanner(session, serviceBanner)
if err != nil {
@@ -411,15 +452,10 @@ func (s *Server) startPTYSession(logger slog.Logger, session ptySession, magicTy
}
if !isQuietLogin(s.fs, session.RawCommand()) {
manifest := s.Manifest.Load()
if manifest != nil {
err := showMOTD(s.fs, session, manifest.MOTDFile)
if err != nil {
logger.Error(ctx, "agent failed to show MOTD", slog.Error(err))
s.metrics.sessionErrors.WithLabelValues(magicTypeLabel, "yes", "motd").Add(1)
}
} else {
logger.Warn(ctx, "metadata lookup failed, unable to show MOTD")
err := showMOTD(s.fs, session, s.config.MOTDFile())
if err != nil {
logger.Error(ctx, "agent failed to show MOTD", slog.Error(err))
s.metrics.sessionErrors.WithLabelValues(magicTypeLabel, "yes", "motd").Add(1)
}
}
@@ -557,7 +593,7 @@ func (s *Server) sftpHandler(logger slog.Logger, session ssh.Session) {
defer server.Close()
err = server.Serve()
if errors.Is(err, io.EOF) {
if err == nil || errors.Is(err, io.EOF) {
// Unless we call `session.Exit(0)` here, the client won't
// receive `exit-status` because `(*sftp.Server).Close()`
// calls `Close()` on the underlying connection (session),
@@ -589,11 +625,6 @@ func (s *Server) CreateCommand(ctx context.Context, script string, env []string)
return nil, xerrors.Errorf("get user shell: %w", err)
}
manifest := s.Manifest.Load()
if manifest == nil {
return nil, xerrors.Errorf("no metadata was provided")
}
// OpenSSH executes all commands with the users current shell.
// We replicate that behavior for IDE support.
caller := "-c"
@@ -638,7 +669,7 @@ func (s *Server) CreateCommand(ctx context.Context, script string, env []string)
}
cmd := pty.CommandContext(ctx, name, args...)
cmd.Dir = manifest.Directory
cmd.Dir = s.config.WorkingDirectory()
// If the metadata directory doesn't exist, we run the command
// in the users home directory.
@@ -652,23 +683,7 @@ func (s *Server) CreateCommand(ctx context.Context, script string, env []string)
cmd.Dir = homedir
}
cmd.Env = append(os.Environ(), env...)
executablePath, err := os.Executable()
if err != nil {
return nil, xerrors.Errorf("getting os executable: %w", err)
}
// Set environment variables reliable detection of being inside a
// Coder workspace.
cmd.Env = append(cmd.Env, "CODER=true")
cmd.Env = append(cmd.Env, "CODER_WORKSPACE_NAME="+manifest.WorkspaceName)
cmd.Env = append(cmd.Env, "CODER_WORKSPACE_AGENT_NAME="+manifest.AgentName)
cmd.Env = append(cmd.Env, fmt.Sprintf("USER=%s", username))
// Git on Windows resolves with UNIX-style paths.
// If using backslashes, it's unable to find the executable.
unixExecutablePath := strings.ReplaceAll(executablePath, "\\", "/")
cmd.Env = append(cmd.Env, fmt.Sprintf(`GIT_SSH_COMMAND=%s gitssh --`, unixExecutablePath))
// Specific Coder subcommands require the agent token exposed!
cmd.Env = append(cmd.Env, fmt.Sprintf("CODER_AGENT_TOKEN=%s", s.AgentToken()))
// Set SSH connection environment variables (these are also set by OpenSSH
// and thus expected to be present by SSH clients). Since the agent does
@@ -679,26 +694,9 @@ func (s *Server) CreateCommand(ctx context.Context, script string, env []string)
cmd.Env = append(cmd.Env, fmt.Sprintf("SSH_CLIENT=%s %s %s", srcAddr, srcPort, dstPort))
cmd.Env = append(cmd.Env, fmt.Sprintf("SSH_CONNECTION=%s %s %s %s", srcAddr, srcPort, dstAddr, dstPort))
// This adds the ports dialog to code-server that enables
// proxying a port dynamically.
cmd.Env = append(cmd.Env, fmt.Sprintf("VSCODE_PROXY_URI=%s", manifest.VSCodePortProxyURI))
// Hide Coder message on code-server's "Getting Started" page
cmd.Env = append(cmd.Env, "CS_DISABLE_GETTING_STARTED_OVERRIDE=true")
// Load environment variables passed via the agent.
// These should override all variables we manually specify.
for envKey, value := range manifest.EnvironmentVariables {
// Expanding environment variables allows for customization
// of the $PATH, among other variables. Customers can prepend
// or append to the $PATH, so allowing expand is required!
cmd.Env = append(cmd.Env, fmt.Sprintf("%s=%s", envKey, os.ExpandEnv(value)))
}
// Agent-level environment variables should take over all!
// This is used for setting agent-specific variables like "CODER_AGENT_TOKEN".
for envKey, value := range s.Env {
cmd.Env = append(cmd.Env, fmt.Sprintf("%s=%s", envKey, value))
cmd.Env, err = s.config.UpdateEnv(cmd.Env)
if err != nil {
return nil, xerrors.Errorf("apply env: %w", err)
}
return cmd, nil
+1 -1
View File
@@ -37,7 +37,7 @@ func Test_sessionStart_orphan(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitMedium)
defer cancel()
logger := slogtest.Make(t, nil)
s, err := NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
+5 -25
View File
@@ -17,14 +17,12 @@ import (
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/atomic"
"go.uber.org/goleak"
"golang.org/x/crypto/ssh"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent/agentssh"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/coder/v2/testutil"
)
@@ -38,14 +36,10 @@ func TestNewServer_ServeClient(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
@@ -83,13 +77,11 @@ func TestNewServer_ExecuteShebang(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
t.Cleanup(func() {
_ = s.Close()
})
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
t.Run("Basic", func(t *testing.T) {
t.Parallel()
@@ -116,14 +108,10 @@ func TestNewServer_CloseActiveConnections(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, &slogtest.Options{IgnoreErrors: true})
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
@@ -171,14 +159,10 @@ func TestNewServer_Signal(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
@@ -240,14 +224,10 @@ func TestNewServer_Signal(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
+3 -3
View File
@@ -32,9 +32,9 @@ func (s *Server) x11Callback(ctx ssh.Context, x11 ssh.X11) bool {
return false
}
err = s.fs.MkdirAll(s.x11SocketDir, 0o700)
err = s.fs.MkdirAll(s.config.X11SocketDir, 0o700)
if err != nil {
s.logger.Warn(ctx, "failed to make the x11 socket dir", slog.F("dir", s.x11SocketDir), slog.Error(err))
s.logger.Warn(ctx, "failed to make the x11 socket dir", slog.F("dir", s.config.X11SocketDir), slog.Error(err))
s.metrics.x11HandlerErrors.WithLabelValues("socker_dir").Add(1)
return false
}
@@ -57,7 +57,7 @@ func (s *Server) x11Handler(ctx ssh.Context, x11 ssh.X11) bool {
return false
}
// We want to overwrite the socket so that subsequent connections will succeed.
socketPath := filepath.Join(s.x11SocketDir, fmt.Sprintf("X%d", x11.ScreenNumber))
socketPath := filepath.Join(s.config.X11SocketDir, fmt.Sprintf("X%d", x11.ScreenNumber))
err := os.Remove(socketPath)
if err != nil && !errors.Is(err, os.ErrNotExist) {
s.logger.Warn(ctx, "failed to remove existing X11 socket", slog.Error(err))
+3 -7
View File
@@ -14,13 +14,11 @@ import (
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/atomic"
gossh "golang.org/x/crypto/ssh"
"cdr.dev/slog"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent/agentssh"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/testutil"
)
@@ -34,14 +32,12 @@ func TestServer_X11(t *testing.T) {
logger := slogtest.Make(t, nil).Leveled(slog.LevelDebug)
fs := afero.NewOsFs()
dir := t.TempDir()
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), fs, 0, dir)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), fs, &agentssh.Config{
X11SocketDir: dir,
})
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
+175 -145
View File
@@ -9,15 +9,19 @@ import (
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/exp/maps"
"golang.org/x/exp/slices"
"golang.org/x/xerrors"
"google.golang.org/protobuf/types/known/durationpb"
"storj.io/drpc"
"storj.io/drpc/drpcmux"
"storj.io/drpc/drpcserver"
"tailscale.com/tailcfg"
"cdr.dev/slog"
agentproto "github.com/coder/coder/v2/agent/proto"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
drpcsdk "github.com/coder/coder/v2/codersdk/drpc"
@@ -26,11 +30,13 @@ import (
"github.com/coder/coder/v2/testutil"
)
const statsInterval = 500 * time.Millisecond
func NewClient(t testing.TB,
logger slog.Logger,
agentID uuid.UUID,
manifest agentsdk.Manifest,
statsChan chan *agentsdk.Stats,
statsChan chan *agentproto.Stats,
coordinator tailnet.Coordinator,
) *Client {
if manifest.AgentID == uuid.Nil {
@@ -39,15 +45,20 @@ func NewClient(t testing.TB,
coordPtr := atomic.Pointer[tailnet.Coordinator]{}
coordPtr.Store(&coordinator)
mux := drpcmux.New()
derpMapUpdates := make(chan *tailcfg.DERPMap)
drpcService := &tailnet.DRPCService{
CoordPtr: &coordPtr,
Logger: logger,
// TODO: handle DERPMap too!
DerpMapUpdateFrequency: time.Hour,
DerpMapFn: func() *tailcfg.DERPMap { panic("not implemented") },
CoordPtr: &coordPtr,
Logger: logger.Named("tailnetsvc"),
DerpMapUpdateFrequency: time.Microsecond,
DerpMapFn: func() *tailcfg.DERPMap { return <-derpMapUpdates },
}
err := proto.DRPCRegisterTailnet(mux, drpcService)
require.NoError(t, err)
mp, err := agentsdk.ProtoFromManifest(manifest)
require.NoError(t, err)
fakeAAPI := NewFakeAgentAPI(t, logger, mp, statsChan)
err = agentproto.DRPCRegisterAgent(mux, fakeAAPI)
require.NoError(t, err)
server := drpcserver.NewWithOptions(mux, drpcserver.Options{
Log: func(err error) {
if xerrors.Is(err, io.EOF) {
@@ -60,145 +71,65 @@ func NewClient(t testing.TB,
t: t,
logger: logger.Named("client"),
agentID: agentID,
manifest: manifest,
statsChan: statsChan,
coordinator: coordinator,
server: server,
derpMapUpdates: make(chan agentsdk.DERPMapUpdate),
fakeAgentAPI: fakeAAPI,
derpMapUpdates: derpMapUpdates,
}
}
type Client struct {
t testing.TB
logger slog.Logger
agentID uuid.UUID
manifest agentsdk.Manifest
metadata map[string]agentsdk.Metadata
statsChan chan *agentsdk.Stats
coordinator tailnet.Coordinator
server *drpcserver.Server
LastWorkspaceAgent func()
PatchWorkspaceLogs func() error
GetServiceBannerFunc func() (codersdk.ServiceBannerConfig, error)
t testing.TB
logger slog.Logger
agentID uuid.UUID
coordinator tailnet.Coordinator
server *drpcserver.Server
fakeAgentAPI *FakeAgentAPI
LastWorkspaceAgent func()
mu sync.Mutex // Protects following.
lifecycleStates []codersdk.WorkspaceAgentLifecycle
startup agentsdk.PostStartupRequest
logs []agentsdk.Log
derpMapUpdates chan agentsdk.DERPMapUpdate
mu sync.Mutex // Protects following.
logs []agentsdk.Log
derpMapUpdates chan *tailcfg.DERPMap
derpMapOnce sync.Once
}
func (c *Client) Manifest(_ context.Context) (agentsdk.Manifest, error) {
return c.manifest, nil
func (*Client) RewriteDERPMap(*tailcfg.DERPMap) {}
func (c *Client) Close() {
c.derpMapOnce.Do(func() { close(c.derpMapUpdates) })
}
func (c *Client) Listen(_ context.Context) (drpc.Conn, error) {
func (c *Client) ConnectRPC(ctx context.Context) (drpc.Conn, error) {
conn, lis := drpcsdk.MemTransportPipe()
closed := make(chan struct{})
c.LastWorkspaceAgent = func() {
_ = conn.Close()
_ = lis.Close()
<-closed
}
c.t.Cleanup(c.LastWorkspaceAgent)
serveCtx, cancel := context.WithCancel(context.Background())
serveCtx, cancel := context.WithCancel(ctx)
c.t.Cleanup(cancel)
auth := tailnet.AgentTunnelAuth{}
streamID := tailnet.StreamID{
Name: "agenttest",
ID: c.agentID,
Auth: auth,
Auth: tailnet.AgentCoordinateeAuth{ID: c.agentID},
}
serveCtx = tailnet.WithStreamID(serveCtx, streamID)
go func() {
_ = c.server.Serve(serveCtx, lis)
close(closed)
}()
return conn, nil
}
func (c *Client) ReportStats(ctx context.Context, _ slog.Logger, statsChan <-chan *agentsdk.Stats, setInterval func(time.Duration)) (io.Closer, error) {
doneCh := make(chan struct{})
ctx, cancel := context.WithCancel(ctx)
go func() {
defer close(doneCh)
setInterval(500 * time.Millisecond)
for {
select {
case <-ctx.Done():
return
case stat := <-statsChan:
select {
case c.statsChan <- stat:
case <-ctx.Done():
return
default:
// We don't want to send old stats.
continue
}
}
}
}()
return closeFunc(func() error {
cancel()
<-doneCh
close(c.statsChan)
return nil
}), nil
}
func (c *Client) GetLifecycleStates() []codersdk.WorkspaceAgentLifecycle {
c.mu.Lock()
defer c.mu.Unlock()
return c.lifecycleStates
return c.fakeAgentAPI.GetLifecycleStates()
}
func (c *Client) PostLifecycle(ctx context.Context, req agentsdk.PostLifecycleRequest) error {
c.mu.Lock()
defer c.mu.Unlock()
c.lifecycleStates = append(c.lifecycleStates, req.State)
c.logger.Debug(ctx, "post lifecycle", slog.F("req", req))
return nil
}
func (c *Client) PostAppHealth(ctx context.Context, req agentsdk.PostAppHealthsRequest) error {
c.logger.Debug(ctx, "post app health", slog.F("req", req))
return nil
}
func (c *Client) GetStartup() agentsdk.PostStartupRequest {
c.mu.Lock()
defer c.mu.Unlock()
return c.startup
func (c *Client) GetStartup() <-chan *agentproto.Startup {
return c.fakeAgentAPI.startupCh
}
func (c *Client) GetMetadata() map[string]agentsdk.Metadata {
c.mu.Lock()
defer c.mu.Unlock()
return maps.Clone(c.metadata)
}
func (c *Client) PostMetadata(ctx context.Context, req agentsdk.PostMetadataRequest) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.metadata == nil {
c.metadata = make(map[string]agentsdk.Metadata)
}
for _, md := range req.Metadata {
c.metadata[md.Key] = md
c.logger.Debug(ctx, "post metadata", slog.F("key", md.Key), slog.F("md", md))
}
return nil
}
func (c *Client) PostStartup(ctx context.Context, startup agentsdk.PostStartupRequest) error {
c.mu.Lock()
defer c.mu.Unlock()
c.startup = startup
c.logger.Debug(ctx, "post startup", slog.F("req", startup))
return nil
return c.fakeAgentAPI.GetMetadata()
}
func (c *Client) GetStartupLogs() []agentsdk.Log {
@@ -207,35 +138,11 @@ func (c *Client) GetStartupLogs() []agentsdk.Log {
return c.logs
}
func (c *Client) PatchLogs(ctx context.Context, logs agentsdk.PatchLogs) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.PatchWorkspaceLogs != nil {
return c.PatchWorkspaceLogs()
}
c.logs = append(c.logs, logs.Logs...)
c.logger.Debug(ctx, "patch startup logs", slog.F("req", logs))
return nil
}
func (c *Client) SetServiceBannerFunc(f func() (codersdk.ServiceBannerConfig, error)) {
c.mu.Lock()
defer c.mu.Unlock()
c.GetServiceBannerFunc = f
c.fakeAgentAPI.SetServiceBannerFunc(f)
}
func (c *Client) GetServiceBanner(ctx context.Context) (codersdk.ServiceBannerConfig, error) {
c.mu.Lock()
defer c.mu.Unlock()
c.logger.Debug(ctx, "get service banner")
if c.GetServiceBannerFunc != nil {
return c.GetServiceBannerFunc()
}
return codersdk.ServiceBannerConfig{}, nil
}
func (c *Client) PushDERPMapUpdate(update agentsdk.DERPMapUpdate) error {
func (c *Client) PushDERPMapUpdate(update *tailcfg.DERPMap) error {
timer := time.NewTimer(testutil.WaitShort)
defer timer.Stop()
select {
@@ -247,16 +154,139 @@ func (c *Client) PushDERPMapUpdate(update agentsdk.DERPMapUpdate) error {
return nil
}
func (c *Client) DERPMapUpdates(_ context.Context) (<-chan agentsdk.DERPMapUpdate, io.Closer, error) {
closed := make(chan struct{})
return c.derpMapUpdates, closeFunc(func() error {
close(closed)
return nil
}), nil
func (c *Client) SetLogsChannel(ch chan<- *agentproto.BatchCreateLogsRequest) {
c.fakeAgentAPI.SetLogsChannel(ch)
}
type closeFunc func() error
type FakeAgentAPI struct {
sync.Mutex
t testing.TB
logger slog.Logger
func (c closeFunc) Close() error {
return c()
manifest *agentproto.Manifest
startupCh chan *agentproto.Startup
statsCh chan *agentproto.Stats
appHealthCh chan *agentproto.BatchUpdateAppHealthRequest
logsCh chan<- *agentproto.BatchCreateLogsRequest
lifecycleStates []codersdk.WorkspaceAgentLifecycle
metadata map[string]agentsdk.Metadata
getServiceBannerFunc func() (codersdk.ServiceBannerConfig, error)
}
func (f *FakeAgentAPI) GetManifest(context.Context, *agentproto.GetManifestRequest) (*agentproto.Manifest, error) {
return f.manifest, nil
}
func (f *FakeAgentAPI) SetServiceBannerFunc(fn func() (codersdk.ServiceBannerConfig, error)) {
f.Lock()
defer f.Unlock()
f.getServiceBannerFunc = fn
f.logger.Info(context.Background(), "updated ServiceBannerFunc")
}
func (f *FakeAgentAPI) GetServiceBanner(context.Context, *agentproto.GetServiceBannerRequest) (*agentproto.ServiceBanner, error) {
f.Lock()
defer f.Unlock()
if f.getServiceBannerFunc == nil {
return &agentproto.ServiceBanner{}, nil
}
sb, err := f.getServiceBannerFunc()
if err != nil {
return nil, err
}
return agentsdk.ProtoFromServiceBanner(sb), nil
}
func (f *FakeAgentAPI) UpdateStats(ctx context.Context, req *agentproto.UpdateStatsRequest) (*agentproto.UpdateStatsResponse, error) {
f.logger.Debug(ctx, "update stats called", slog.F("req", req))
// empty request is sent to get the interval; but our tests don't want empty stats requests
if req.Stats != nil {
f.statsCh <- req.Stats
}
return &agentproto.UpdateStatsResponse{ReportInterval: durationpb.New(statsInterval)}, nil
}
func (f *FakeAgentAPI) GetLifecycleStates() []codersdk.WorkspaceAgentLifecycle {
f.Lock()
defer f.Unlock()
return slices.Clone(f.lifecycleStates)
}
func (f *FakeAgentAPI) UpdateLifecycle(_ context.Context, req *agentproto.UpdateLifecycleRequest) (*agentproto.Lifecycle, error) {
f.Lock()
defer f.Unlock()
s, err := agentsdk.LifecycleStateFromProto(req.GetLifecycle().GetState())
if assert.NoError(f.t, err) {
f.lifecycleStates = append(f.lifecycleStates, s)
}
return req.GetLifecycle(), nil
}
func (f *FakeAgentAPI) BatchUpdateAppHealths(ctx context.Context, req *agentproto.BatchUpdateAppHealthRequest) (*agentproto.BatchUpdateAppHealthResponse, error) {
f.logger.Debug(ctx, "batch update app health", slog.F("req", req))
f.appHealthCh <- req
return &agentproto.BatchUpdateAppHealthResponse{}, nil
}
func (f *FakeAgentAPI) AppHealthCh() <-chan *agentproto.BatchUpdateAppHealthRequest {
return f.appHealthCh
}
func (f *FakeAgentAPI) UpdateStartup(_ context.Context, req *agentproto.UpdateStartupRequest) (*agentproto.Startup, error) {
f.startupCh <- req.GetStartup()
return req.GetStartup(), nil
}
func (f *FakeAgentAPI) GetMetadata() map[string]agentsdk.Metadata {
f.Lock()
defer f.Unlock()
return maps.Clone(f.metadata)
}
func (f *FakeAgentAPI) BatchUpdateMetadata(ctx context.Context, req *agentproto.BatchUpdateMetadataRequest) (*agentproto.BatchUpdateMetadataResponse, error) {
f.Lock()
defer f.Unlock()
if f.metadata == nil {
f.metadata = make(map[string]agentsdk.Metadata)
}
for _, md := range req.Metadata {
smd := agentsdk.MetadataFromProto(md)
f.metadata[md.Key] = smd
f.logger.Debug(ctx, "post metadata", slog.F("key", md.Key), slog.F("md", md))
}
return &agentproto.BatchUpdateMetadataResponse{}, nil
}
func (f *FakeAgentAPI) SetLogsChannel(ch chan<- *agentproto.BatchCreateLogsRequest) {
f.Lock()
defer f.Unlock()
f.logsCh = ch
}
func (f *FakeAgentAPI) BatchCreateLogs(ctx context.Context, req *agentproto.BatchCreateLogsRequest) (*agentproto.BatchCreateLogsResponse, error) {
f.logger.Info(ctx, "batch create logs called", slog.F("req", req))
f.Lock()
ch := f.logsCh
f.Unlock()
if ch != nil {
select {
case <-ctx.Done():
return nil, ctx.Err()
case ch <- req:
// ok
}
}
return &agentproto.BatchCreateLogsResponse{}, nil
}
func NewFakeAgentAPI(t testing.TB, logger slog.Logger, manifest *agentproto.Manifest, statsCh chan *agentproto.Stats) *FakeAgentAPI {
return &FakeAgentAPI{
t: t,
logger: logger.Named("FakeAgentAPI"),
manifest: manifest,
statsCh: statsCh,
startupCh: make(chan *agentproto.Startup, 100),
appHealthCh: make(chan *agentproto.BatchUpdateAppHealthRequest, 100),
}
}
+6
View File
@@ -35,7 +35,13 @@ func (a *agent) apiHandler() http.Handler {
ignorePorts: cpy,
cacheDuration: cacheDuration,
}
promHandler := PrometheusMetricsHandler(a.prometheusRegistry, a.logger)
r.Get("/api/v0/listening-ports", lp.handler)
r.Get("/debug/logs", a.HandleHTTPDebugLogs)
r.Get("/debug/magicsock", a.HandleHTTPDebugMagicsock)
r.Get("/debug/magicsock/debug-logging/{state}", a.HandleHTTPMagicsockDebugLoggingState)
r.Get("/debug/manifest", a.HandleHTTPDebugManifest)
r.Get("/debug/prometheus", promHandler.ServeHTTP)
return r
}
+16 -1
View File
@@ -26,7 +26,12 @@ type WorkspaceAppHealthReporter func(ctx context.Context)
// NewWorkspaceAppHealthReporter creates a WorkspaceAppHealthReporter that reports app health to coderd.
func NewWorkspaceAppHealthReporter(logger slog.Logger, apps []codersdk.WorkspaceApp, postWorkspaceAgentAppHealth PostWorkspaceAgentAppHealth) WorkspaceAppHealthReporter {
logger = logger.Named("apphealth")
runHealthcheckLoop := func(ctx context.Context) error {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// no need to run this loop if no apps for this workspace.
if len(apps) == 0 {
return nil
@@ -87,6 +92,7 @@ func NewWorkspaceAppHealthReporter(logger slog.Logger, apps []codersdk.Workspace
return nil
}()
if err != nil {
nowUnhealthy := false
mu.Lock()
if failures[app.ID] < int(app.Healthcheck.Threshold) {
// increment the failure count and keep status the same.
@@ -96,14 +102,21 @@ func NewWorkspaceAppHealthReporter(logger slog.Logger, apps []codersdk.Workspace
// set to unhealthy if we hit the failure threshold.
// we stop incrementing at the threshold to prevent the failure value from increasing forever.
health[app.ID] = codersdk.WorkspaceAppHealthUnhealthy
nowUnhealthy = true
}
mu.Unlock()
logger.Debug(ctx, "error checking app health",
slog.F("id", app.ID.String()),
slog.F("slug", app.Slug),
slog.F("now_unhealthy", nowUnhealthy), slog.Error(err),
)
} else {
mu.Lock()
// we only need one successful health check to be considered healthy.
health[app.ID] = codersdk.WorkspaceAppHealthHealthy
failures[app.ID] = 0
mu.Unlock()
logger.Debug(ctx, "workspace app healthy", slog.F("id", app.ID.String()), slog.F("slug", app.Slug))
}
t.Reset(time.Duration(app.Healthcheck.Interval) * time.Second)
@@ -137,7 +150,9 @@ func NewWorkspaceAppHealthReporter(logger slog.Logger, apps []codersdk.Workspace
Healths: lastHealth,
})
if err != nil {
logger.Error(ctx, "failed to report workspace app stat", slog.Error(err))
logger.Error(ctx, "failed to report workspace app health", slog.Error(err))
} else {
logger.Debug(ctx, "sent workspace app health", slog.F("health", lastHealth))
}
}
}
+56 -14
View File
@@ -4,16 +4,21 @@ import (
"context"
"net/http"
"net/http/httptest"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"cdr.dev/slog"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent"
"github.com/coder/coder/v2/agent/agenttest"
"github.com/coder/coder/v2/agent/proto"
"github.com/coder/coder/v2/coderd/httpapi"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
@@ -40,12 +45,23 @@ func TestAppHealth_Healthy(t *testing.T) {
},
Health: codersdk.WorkspaceAppHealthInitializing,
},
{
Slug: "app3",
Healthcheck: codersdk.Healthcheck{
Interval: 2,
Threshold: 1,
},
Health: codersdk.WorkspaceAppHealthInitializing,
},
}
handlers := []http.Handler{
nil,
http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
httpapi.Write(r.Context(), w, http.StatusOK, nil)
}),
http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
httpapi.Write(r.Context(), w, http.StatusOK, nil)
}),
}
getApps, closeFn := setupAppReporter(ctx, t, apps, handlers)
defer closeFn()
@@ -58,7 +74,7 @@ func TestAppHealth_Healthy(t *testing.T) {
return false
}
return apps[1].Health == codersdk.WorkspaceAppHealthHealthy
return apps[1].Health == codersdk.WorkspaceAppHealthHealthy && apps[2].Health == codersdk.WorkspaceAppHealthHealthy
}, testutil.WaitLong, testutil.IntervalSlow)
}
@@ -163,6 +179,12 @@ func TestAppHealth_NotSpamming(t *testing.T) {
func setupAppReporter(ctx context.Context, t *testing.T, apps []codersdk.WorkspaceApp, handlers []http.Handler) (agent.WorkspaceAgentApps, func()) {
closers := []func(){}
for i, app := range apps {
if app.ID == uuid.Nil {
app.ID = uuid.New()
apps[i] = app
}
}
for i, handler := range handlers {
if handler == nil {
continue
@@ -181,23 +203,43 @@ func setupAppReporter(ctx context.Context, t *testing.T, apps []codersdk.Workspa
var newApps []codersdk.WorkspaceApp
return append(newApps, apps...), nil
}
postWorkspaceAgentAppHealth := func(_ context.Context, req agentsdk.PostAppHealthsRequest) error {
mu.Lock()
for id, health := range req.Healths {
for i, app := range apps {
if app.ID != id {
continue
// We don't care about manifest or stats in this test since it's not using
// a full agent and these RPCs won't get called.
//
// We use a proper fake agent API so we can test the conversion code and the
// request code as well. Before we were bypassing these by using a custom
// post function.
fakeAAPI := agenttest.NewFakeAgentAPI(t, slogtest.Make(t, nil), nil, nil)
// Process events from the channel and update the health of the apps.
go func() {
appHealthCh := fakeAAPI.AppHealthCh()
for {
select {
case <-ctx.Done():
return
case req := <-appHealthCh:
mu.Lock()
for _, update := range req.Updates {
updateID, err := uuid.FromBytes(update.Id)
assert.NoError(t, err)
updateHealth := codersdk.WorkspaceAppHealth(strings.ToLower(proto.AppHealth_name[int32(update.Health)]))
for i, app := range apps {
if app.ID != updateID {
continue
}
app.Health = updateHealth
apps[i] = app
}
}
app.Health = health
apps[i] = app
mu.Unlock()
}
}
mu.Unlock()
}()
return nil
}
go agent.NewWorkspaceAppHealthReporter(slogtest.Make(t, nil).Leveled(slog.LevelDebug), apps, postWorkspaceAgentAppHealth)(ctx)
go agent.NewWorkspaceAppHealthReporter(slogtest.Make(t, nil).Leveled(slog.LevelDebug), apps, agentsdk.AppHealthPoster(fakeAAPI))(ctx)
return workspaceAgentApps, func() {
for _, closeFn := range closers {
+14 -15
View File
@@ -10,8 +10,7 @@ import (
"tailscale.com/util/clientmetric"
"cdr.dev/slog"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/agent/proto"
)
type agentMetrics struct {
@@ -53,8 +52,8 @@ func newAgentMetrics(registerer prometheus.Registerer) *agentMetrics {
}
}
func (a *agent) collectMetrics(ctx context.Context) []agentsdk.AgentMetric {
var collected []agentsdk.AgentMetric
func (a *agent) collectMetrics(ctx context.Context) []*proto.Stats_Metric {
var collected []*proto.Stats_Metric
// Tailscale internal metrics
metrics := clientmetric.Metrics()
@@ -63,7 +62,7 @@ func (a *agent) collectMetrics(ctx context.Context) []agentsdk.AgentMetric {
continue
}
collected = append(collected, agentsdk.AgentMetric{
collected = append(collected, &proto.Stats_Metric{
Name: m.Name(),
Type: asMetricType(m.Type()),
Value: float64(m.Value()),
@@ -81,16 +80,16 @@ func (a *agent) collectMetrics(ctx context.Context) []agentsdk.AgentMetric {
labels := toAgentMetricLabels(metric.Label)
if metric.Counter != nil {
collected = append(collected, agentsdk.AgentMetric{
collected = append(collected, &proto.Stats_Metric{
Name: metricFamily.GetName(),
Type: agentsdk.AgentMetricTypeCounter,
Type: proto.Stats_Metric_COUNTER,
Value: metric.Counter.GetValue(),
Labels: labels,
})
} else if metric.Gauge != nil {
collected = append(collected, agentsdk.AgentMetric{
collected = append(collected, &proto.Stats_Metric{
Name: metricFamily.GetName(),
Type: agentsdk.AgentMetricTypeGauge,
Type: proto.Stats_Metric_GAUGE,
Value: metric.Gauge.GetValue(),
Labels: labels,
})
@@ -102,14 +101,14 @@ func (a *agent) collectMetrics(ctx context.Context) []agentsdk.AgentMetric {
return collected
}
func toAgentMetricLabels(metricLabels []*prompb.LabelPair) []agentsdk.AgentMetricLabel {
func toAgentMetricLabels(metricLabels []*prompb.LabelPair) []*proto.Stats_Metric_Label {
if len(metricLabels) == 0 {
return nil
}
labels := make([]agentsdk.AgentMetricLabel, 0, len(metricLabels))
labels := make([]*proto.Stats_Metric_Label, 0, len(metricLabels))
for _, metricLabel := range metricLabels {
labels = append(labels, agentsdk.AgentMetricLabel{
labels = append(labels, &proto.Stats_Metric_Label{
Name: metricLabel.GetName(),
Value: metricLabel.GetValue(),
})
@@ -130,12 +129,12 @@ func isIgnoredMetric(metricName string) bool {
return false
}
func asMetricType(typ clientmetric.Type) agentsdk.AgentMetricType {
func asMetricType(typ clientmetric.Type) proto.Stats_Metric_Type {
switch typ {
case clientmetric.TypeGauge:
return agentsdk.AgentMetricTypeGauge
return proto.Stats_Metric_GAUGE
case clientmetric.TypeCounter:
return agentsdk.AgentMetricTypeCounter
return proto.Stats_Metric_COUNTER
default:
panic(fmt.Sprintf("unknown metric type: %d", typ))
}
+2 -1
View File
@@ -9,6 +9,7 @@ import (
"golang.org/x/xerrors"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
)
func (lp *listeningPortsHandler) getListeningPorts() ([]codersdk.WorkspaceAgentListeningPort, error) {
@@ -32,7 +33,7 @@ func (lp *listeningPortsHandler) getListeningPorts() ([]codersdk.WorkspaceAgentL
seen := make(map[uint16]struct{}, len(tabs))
ports := []codersdk.WorkspaceAgentListeningPort{}
for _, tab := range tabs {
if tab.LocalAddr == nil || tab.LocalAddr.Port < codersdk.WorkspaceAgentMinimumListeningPort {
if tab.LocalAddr == nil || tab.LocalAddr.Port < workspacesdk.AgentMinimumListeningPort {
continue
}
+70 -58
View File
@@ -1816,6 +1816,8 @@ type BatchCreateLogsResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
LogLimitExceeded bool `protobuf:"varint,1,opt,name=log_limit_exceeded,json=logLimitExceeded,proto3" json:"log_limit_exceeded,omitempty"`
}
func (x *BatchCreateLogsResponse) Reset() {
@@ -1850,6 +1852,13 @@ func (*BatchCreateLogsResponse) Descriptor() ([]byte, []int) {
return file_agent_proto_agent_proto_rawDescGZIP(), []int{21}
}
func (x *BatchCreateLogsResponse) GetLogLimitExceeded() bool {
if x != nil {
return x.LogLimitExceeded
}
return false
}
type WorkspaceApp_Healthcheck struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -2580,66 +2589,69 @@ var file_agent_proto_agent_proto_rawDesc = []byte{
0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0b, 0x6c, 0x6f, 0x67, 0x53, 0x6f, 0x75, 0x72, 0x63, 0x65,
0x49, 0x64, 0x12, 0x27, 0x0a, 0x04, 0x6c, 0x6f, 0x67, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b,
0x32, 0x13, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76,
0x32, 0x2e, 0x4c, 0x6f, 0x67, 0x52, 0x04, 0x6c, 0x6f, 0x67, 0x73, 0x22, 0x19, 0x0a, 0x17, 0x42,
0x32, 0x2e, 0x4c, 0x6f, 0x67, 0x52, 0x04, 0x6c, 0x6f, 0x67, 0x73, 0x22, 0x47, 0x0a, 0x17, 0x42,
0x61, 0x74, 0x63, 0x68, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x4c, 0x6f, 0x67, 0x73, 0x52, 0x65,
0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x2a, 0x63, 0x0a, 0x09, 0x41, 0x70, 0x70, 0x48, 0x65, 0x61,
0x6c, 0x74, 0x68, 0x12, 0x1a, 0x0a, 0x16, 0x41, 0x50, 0x50, 0x5f, 0x48, 0x45, 0x41, 0x4c, 0x54,
0x48, 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12,
0x0c, 0x0a, 0x08, 0x44, 0x49, 0x53, 0x41, 0x42, 0x4c, 0x45, 0x44, 0x10, 0x01, 0x12, 0x10, 0x0a,
0x0c, 0x49, 0x4e, 0x49, 0x54, 0x49, 0x41, 0x4c, 0x49, 0x5a, 0x49, 0x4e, 0x47, 0x10, 0x02, 0x12,
0x0b, 0x0a, 0x07, 0x48, 0x45, 0x41, 0x4c, 0x54, 0x48, 0x59, 0x10, 0x03, 0x12, 0x0d, 0x0a, 0x09,
0x55, 0x4e, 0x48, 0x45, 0x41, 0x4c, 0x54, 0x48, 0x59, 0x10, 0x04, 0x32, 0xf6, 0x05, 0x0a, 0x05,
0x41, 0x67, 0x65, 0x6e, 0x74, 0x12, 0x4b, 0x0a, 0x0b, 0x47, 0x65, 0x74, 0x4d, 0x61, 0x6e, 0x69,
0x66, 0x65, 0x73, 0x74, 0x12, 0x22, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65,
0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x47, 0x65, 0x74, 0x4d, 0x61, 0x6e, 0x69, 0x66, 0x65, 0x73,
0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x18, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72,
0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x4d, 0x61, 0x6e, 0x69, 0x66, 0x65,
0x73, 0x74, 0x12, 0x5a, 0x0a, 0x10, 0x47, 0x65, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65,
0x42, 0x61, 0x6e, 0x6e, 0x65, 0x72, 0x12, 0x27, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61,
0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x47, 0x65, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69,
0x63, 0x65, 0x42, 0x61, 0x6e, 0x6e, 0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
0x1d, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32,
0x2e, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x42, 0x61, 0x6e, 0x6e, 0x65, 0x72, 0x12, 0x56,
0x0a, 0x0b, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x74, 0x73, 0x12, 0x22, 0x2e,
0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x55,
0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x1a, 0x23, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e,
0x76, 0x32, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x74, 0x73, 0x52, 0x65,
0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x54, 0x0a, 0x0f, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65,
0x4c, 0x69, 0x66, 0x65, 0x63, 0x79, 0x63, 0x6c, 0x65, 0x12, 0x26, 0x2e, 0x63, 0x6f, 0x64, 0x65,
0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74,
0x65, 0x4c, 0x69, 0x66, 0x65, 0x63, 0x79, 0x63, 0x6c, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x1a, 0x19, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e,
0x76, 0x32, 0x2e, 0x4c, 0x69, 0x66, 0x65, 0x63, 0x79, 0x63, 0x6c, 0x65, 0x12, 0x72, 0x0a, 0x15,
0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x41, 0x70, 0x70, 0x48, 0x65,
0x61, 0x6c, 0x74, 0x68, 0x73, 0x12, 0x2b, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67,
0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61,
0x74, 0x65, 0x41, 0x70, 0x70, 0x48, 0x65, 0x61, 0x6c, 0x74, 0x68, 0x52, 0x65, 0x71, 0x75, 0x65,
0x73, 0x74, 0x1a, 0x2c, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74,
0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x41,
0x70, 0x70, 0x48, 0x65, 0x61, 0x6c, 0x74, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x12, 0x4e, 0x0a, 0x0d, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x72, 0x74, 0x75,
0x70, 0x12, 0x24, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e,
0x76, 0x32, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x72, 0x74, 0x75, 0x70,
0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e,
0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x53, 0x74, 0x61, 0x72, 0x74, 0x75, 0x70,
0x12, 0x6e, 0x0a, 0x13, 0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4d,
0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x12, 0x2a, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e,
0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70,
0x64, 0x61, 0x74, 0x65, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75,
0x65, 0x73, 0x74, 0x1a, 0x2b, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e,
0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2c, 0x0a, 0x12, 0x6c, 0x6f, 0x67, 0x5f, 0x6c, 0x69,
0x6d, 0x69, 0x74, 0x5f, 0x65, 0x78, 0x63, 0x65, 0x65, 0x64, 0x65, 0x64, 0x18, 0x01, 0x20, 0x01,
0x28, 0x08, 0x52, 0x10, 0x6c, 0x6f, 0x67, 0x4c, 0x69, 0x6d, 0x69, 0x74, 0x45, 0x78, 0x63, 0x65,
0x65, 0x64, 0x65, 0x64, 0x2a, 0x63, 0x0a, 0x09, 0x41, 0x70, 0x70, 0x48, 0x65, 0x61, 0x6c, 0x74,
0x68, 0x12, 0x1a, 0x0a, 0x16, 0x41, 0x50, 0x50, 0x5f, 0x48, 0x45, 0x41, 0x4c, 0x54, 0x48, 0x5f,
0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x0c, 0x0a,
0x08, 0x44, 0x49, 0x53, 0x41, 0x42, 0x4c, 0x45, 0x44, 0x10, 0x01, 0x12, 0x10, 0x0a, 0x0c, 0x49,
0x4e, 0x49, 0x54, 0x49, 0x41, 0x4c, 0x49, 0x5a, 0x49, 0x4e, 0x47, 0x10, 0x02, 0x12, 0x0b, 0x0a,
0x07, 0x48, 0x45, 0x41, 0x4c, 0x54, 0x48, 0x59, 0x10, 0x03, 0x12, 0x0d, 0x0a, 0x09, 0x55, 0x4e,
0x48, 0x45, 0x41, 0x4c, 0x54, 0x48, 0x59, 0x10, 0x04, 0x32, 0xf6, 0x05, 0x0a, 0x05, 0x41, 0x67,
0x65, 0x6e, 0x74, 0x12, 0x4b, 0x0a, 0x0b, 0x47, 0x65, 0x74, 0x4d, 0x61, 0x6e, 0x69, 0x66, 0x65,
0x73, 0x74, 0x12, 0x22, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74,
0x2e, 0x76, 0x32, 0x2e, 0x47, 0x65, 0x74, 0x4d, 0x61, 0x6e, 0x69, 0x66, 0x65, 0x73, 0x74, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x18, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61,
0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x4d, 0x61, 0x6e, 0x69, 0x66, 0x65, 0x73, 0x74,
0x12, 0x5a, 0x0a, 0x10, 0x47, 0x65, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x42, 0x61,
0x6e, 0x6e, 0x65, 0x72, 0x12, 0x27, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65,
0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x47, 0x65, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65,
0x42, 0x61, 0x6e, 0x6e, 0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d, 0x2e,
0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x53,
0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x42, 0x61, 0x6e, 0x6e, 0x65, 0x72, 0x12, 0x56, 0x0a, 0x0b,
0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x74, 0x73, 0x12, 0x22, 0x2e, 0x63, 0x6f,
0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x55, 0x70, 0x64,
0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
0x23, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32,
0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x74, 0x73, 0x52, 0x65, 0x73, 0x70,
0x6f, 0x6e, 0x73, 0x65, 0x12, 0x54, 0x0a, 0x0f, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4c, 0x69,
0x66, 0x65, 0x63, 0x79, 0x63, 0x6c, 0x65, 0x12, 0x26, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e,
0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4c,
0x69, 0x66, 0x65, 0x63, 0x79, 0x63, 0x6c, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
0x19, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32,
0x2e, 0x4c, 0x69, 0x66, 0x65, 0x63, 0x79, 0x63, 0x6c, 0x65, 0x12, 0x72, 0x0a, 0x15, 0x42, 0x61,
0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x41, 0x70, 0x70, 0x48, 0x65, 0x61, 0x6c,
0x74, 0x68, 0x73, 0x12, 0x2b, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e,
0x74, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65,
0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x12, 0x62, 0x0a, 0x0f, 0x42, 0x61, 0x74, 0x63, 0x68, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x4c,
0x6f, 0x67, 0x73, 0x12, 0x26, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e,
0x74, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65,
0x4c, 0x6f, 0x67, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x27, 0x2e, 0x63, 0x6f,
0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x74,
0x63, 0x68, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x4c, 0x6f, 0x67, 0x73, 0x52, 0x65, 0x73, 0x70,
0x6f, 0x6e, 0x73, 0x65, 0x42, 0x27, 0x5a, 0x25, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63,
0x6f, 0x6d, 0x2f, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2f, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2f, 0x76,
0x32, 0x2f, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x06, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x33,
0x41, 0x70, 0x70, 0x48, 0x65, 0x61, 0x6c, 0x74, 0x68, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,
0x1a, 0x2c, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76,
0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x41, 0x70, 0x70,
0x48, 0x65, 0x61, 0x6c, 0x74, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x4e,
0x0a, 0x0d, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x72, 0x74, 0x75, 0x70, 0x12,
0x24, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32,
0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x61, 0x72, 0x74, 0x75, 0x70, 0x52, 0x65,
0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67,
0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x53, 0x74, 0x61, 0x72, 0x74, 0x75, 0x70, 0x12, 0x6e,
0x0a, 0x13, 0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4d, 0x65, 0x74,
0x61, 0x64, 0x61, 0x74, 0x61, 0x12, 0x2a, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67,
0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61,
0x74, 0x65, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x1a, 0x2b, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e,
0x76, 0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4d, 0x65,
0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x62,
0x0a, 0x0f, 0x42, 0x61, 0x74, 0x63, 0x68, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x4c, 0x6f, 0x67,
0x73, 0x12, 0x26, 0x2e, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e,
0x76, 0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x4c, 0x6f,
0x67, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x27, 0x2e, 0x63, 0x6f, 0x64, 0x65,
0x72, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x74, 0x63, 0x68,
0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x4c, 0x6f, 0x67, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e,
0x73, 0x65, 0x42, 0x27, 0x5a, 0x25, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d,
0x2f, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2f, 0x63, 0x6f, 0x64, 0x65, 0x72, 0x2f, 0x76, 0x32, 0x2f,
0x61, 0x67, 0x65, 0x6e, 0x74, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x06, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x33,
}
var (
+3 -1
View File
@@ -247,7 +247,9 @@ message BatchCreateLogsRequest {
repeated Log logs = 2;
}
message BatchCreateLogsResponse {}
message BatchCreateLogsResponse {
bool log_limit_exceeded = 1;
}
service Agent {
rpc GetManifest(GetManifestRequest) returns (Manifest);
-106
View File
@@ -1,106 +0,0 @@
package proto
import (
"strings"
"github.com/google/uuid"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/codersdk"
)
func SDKAgentMetadataDescriptionsFromProto(descriptions []*WorkspaceAgentMetadata_Description) []codersdk.WorkspaceAgentMetadataDescription {
ret := make([]codersdk.WorkspaceAgentMetadataDescription, len(descriptions))
for i, description := range descriptions {
ret[i] = SDKAgentMetadataDescriptionFromProto(description)
}
return ret
}
func SDKAgentMetadataDescriptionFromProto(description *WorkspaceAgentMetadata_Description) codersdk.WorkspaceAgentMetadataDescription {
return codersdk.WorkspaceAgentMetadataDescription{
DisplayName: description.DisplayName,
Key: description.Key,
Script: description.Script,
Interval: int64(description.Interval.AsDuration()),
Timeout: int64(description.Timeout.AsDuration()),
}
}
func SDKAgentScriptsFromProto(protoScripts []*WorkspaceAgentScript) ([]codersdk.WorkspaceAgentScript, error) {
ret := make([]codersdk.WorkspaceAgentScript, len(protoScripts))
for i, protoScript := range protoScripts {
app, err := SDKAgentScriptFromProto(protoScript)
if err != nil {
return nil, xerrors.Errorf("parse script %v: %w", i, err)
}
ret[i] = app
}
return ret, nil
}
func SDKAgentScriptFromProto(protoScript *WorkspaceAgentScript) (codersdk.WorkspaceAgentScript, error) {
id, err := uuid.FromBytes(protoScript.LogSourceId)
if err != nil {
return codersdk.WorkspaceAgentScript{}, xerrors.Errorf("parse id: %w", err)
}
return codersdk.WorkspaceAgentScript{
LogSourceID: id,
LogPath: protoScript.LogPath,
Script: protoScript.Script,
Cron: protoScript.Cron,
RunOnStart: protoScript.RunOnStart,
RunOnStop: protoScript.RunOnStop,
StartBlocksLogin: protoScript.StartBlocksLogin,
Timeout: protoScript.Timeout.AsDuration(),
}, nil
}
func SDKAppsFromProto(protoApps []*WorkspaceApp) ([]codersdk.WorkspaceApp, error) {
ret := make([]codersdk.WorkspaceApp, len(protoApps))
for i, protoApp := range protoApps {
app, err := SDKAppFromProto(protoApp)
if err != nil {
return nil, xerrors.Errorf("parse app %v (%q): %w", i, protoApp.Slug, err)
}
ret[i] = app
}
return ret, nil
}
func SDKAppFromProto(protoApp *WorkspaceApp) (codersdk.WorkspaceApp, error) {
id, err := uuid.FromBytes(protoApp.Id)
if err != nil {
return codersdk.WorkspaceApp{}, xerrors.Errorf("parse id: %w", err)
}
var sharingLevel codersdk.WorkspaceAppSharingLevel = codersdk.WorkspaceAppSharingLevel(strings.ToLower(protoApp.SharingLevel.String()))
if _, ok := codersdk.MapWorkspaceAppSharingLevels[sharingLevel]; !ok {
return codersdk.WorkspaceApp{}, xerrors.Errorf("unknown app sharing level: %v (%q)", protoApp.SharingLevel, protoApp.SharingLevel.String())
}
var health codersdk.WorkspaceAppHealth = codersdk.WorkspaceAppHealth(strings.ToLower(protoApp.Health.String()))
if _, ok := codersdk.MapWorkspaceAppHealths[health]; !ok {
return codersdk.WorkspaceApp{}, xerrors.Errorf("unknown app health: %v (%q)", protoApp.Health, protoApp.Health.String())
}
return codersdk.WorkspaceApp{
ID: id,
URL: protoApp.Url,
External: protoApp.External,
Slug: protoApp.Slug,
DisplayName: protoApp.DisplayName,
Command: protoApp.Command,
Icon: protoApp.Icon,
Subdomain: protoApp.Subdomain,
SubdomainName: protoApp.SubdomainName,
SharingLevel: sharingLevel,
Healthcheck: codersdk.Healthcheck{
URL: protoApp.Healthcheck.Url,
Interval: int32(protoApp.Healthcheck.Interval.AsDuration().Seconds()),
Threshold: protoApp.Healthcheck.Threshold,
},
Health: health,
}, nil
}
+10
View File
@@ -0,0 +1,10 @@
package proto
import (
"github.com/coder/coder/v2/tailnet/proto"
)
// CurrentVersion is the current version of the agent API. It is tied to the
// tailnet API version to avoid confusion, since agents connect to the tailnet
// API over the same websocket.
var CurrentVersion = proto.CurrentVersion
+2 -3
View File
@@ -14,8 +14,7 @@ import (
"golang.org/x/xerrors"
"cdr.dev/slog"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
"github.com/coder/coder/v2/pty"
)
@@ -197,7 +196,7 @@ func (s *ptyState) waitForStateOrContext(ctx context.Context, state State) (Stat
func readConnLoop(ctx context.Context, conn net.Conn, ptty pty.PTYCmd, metrics *prometheus.CounterVec, logger slog.Logger) {
decoder := json.NewDecoder(conn)
for {
var req codersdk.ReconnectingPTYRequest
var req workspacesdk.ReconnectingPTYRequest
err := decoder.Decode(&req)
if xerrors.Is(err, io.EOF) {
return
+7
View File
@@ -81,6 +81,13 @@ func newScreen(ctx context.Context, cmd *pty.Cmd, options *Options, logger slog.
rpty.id = hex.EncodeToString(buf)
settings := []string{
// Disable the startup message that appears for five seconds.
"startup_message off",
// Some message are hard-coded, the best we can do is set msgwait to 0
// which seems to hide them. This can happen for example if screen shows
// the version message when starting up.
"msgminwait 0",
"msgwait 0",
// Tell screen not to handle motion for xterm* terminals which allows
// scrolling the terminal via the mouse wheel or scroll bar (by default
// screen uses it to cycle through the command history). There does not
+126
View File
@@ -0,0 +1,126 @@
package agent
import (
"context"
"sync"
"time"
"golang.org/x/xerrors"
"tailscale.com/types/netlogtype"
"cdr.dev/slog"
"github.com/coder/coder/v2/agent/proto"
)
const maxConns = 2048
type networkStatsSource interface {
SetConnStatsCallback(maxPeriod time.Duration, maxConns int, dump func(start, end time.Time, virtual, physical map[netlogtype.Connection]netlogtype.Counts))
}
type statsCollector interface {
Collect(ctx context.Context, networkStats map[netlogtype.Connection]netlogtype.Counts) *proto.Stats
}
type statsDest interface {
UpdateStats(ctx context.Context, req *proto.UpdateStatsRequest) (*proto.UpdateStatsResponse, error)
}
// statsReporter is a subcomponent of the agent that handles registering the stats callback on the
// networkStatsSource (tailnet.Conn in prod), handling the callback, calling back to the
// statsCollector (agent in prod) to collect additional stats, then sending the update to the
// statsDest (agent API in prod)
type statsReporter struct {
*sync.Cond
networkStats *map[netlogtype.Connection]netlogtype.Counts
unreported bool
lastInterval time.Duration
source networkStatsSource
collector statsCollector
logger slog.Logger
}
func newStatsReporter(logger slog.Logger, source networkStatsSource, collector statsCollector) *statsReporter {
return &statsReporter{
Cond: sync.NewCond(&sync.Mutex{}),
logger: logger,
source: source,
collector: collector,
}
}
func (s *statsReporter) callback(_, _ time.Time, virtual, _ map[netlogtype.Connection]netlogtype.Counts) {
s.L.Lock()
defer s.L.Unlock()
s.logger.Debug(context.Background(), "got stats callback")
s.networkStats = &virtual
s.unreported = true
s.Broadcast()
}
// reportLoop programs the source (tailnet.Conn) to send it stats via the
// callback, then reports them to the dest.
//
// It's intended to be called within the larger retry loop that establishes a
// connection to the agent API, then passes that connection to go routines like
// this that use it. There is no retry and we fail on the first error since
// this will be inside a larger retry loop.
func (s *statsReporter) reportLoop(ctx context.Context, dest statsDest) error {
// send an initial, blank report to get the interval
resp, err := dest.UpdateStats(ctx, &proto.UpdateStatsRequest{})
if err != nil {
return xerrors.Errorf("initial update: %w", err)
}
s.lastInterval = resp.ReportInterval.AsDuration()
s.source.SetConnStatsCallback(s.lastInterval, maxConns, s.callback)
// use a separate goroutine to monitor the context so that we notice immediately, rather than
// waiting for the next callback (which might never come if we are closing!)
ctxDone := false
go func() {
<-ctx.Done()
s.L.Lock()
defer s.L.Unlock()
ctxDone = true
s.Broadcast()
}()
defer s.logger.Debug(ctx, "reportLoop exiting")
s.L.Lock()
defer s.L.Unlock()
for {
for !s.unreported && !ctxDone {
s.Wait()
}
if ctxDone {
return nil
}
networkStats := *s.networkStats
s.unreported = false
if err = s.reportLocked(ctx, dest, networkStats); err != nil {
return xerrors.Errorf("report stats: %w", err)
}
}
}
func (s *statsReporter) reportLocked(
ctx context.Context, dest statsDest, networkStats map[netlogtype.Connection]netlogtype.Counts,
) error {
// here we want to do our collecting/reporting while it is unlocked, but then relock
// when we return to reportLoop.
s.L.Unlock()
defer s.L.Lock()
stats := s.collector.Collect(ctx, networkStats)
resp, err := dest.UpdateStats(ctx, &proto.UpdateStatsRequest{Stats: stats})
if err != nil {
return err
}
interval := resp.GetReportInterval().AsDuration()
if interval != s.lastInterval {
s.logger.Info(ctx, "new stats report interval", slog.F("interval", interval))
s.lastInterval = interval
s.source.SetConnStatsCallback(s.lastInterval, maxConns, s.callback)
}
return nil
}
+271
View File
@@ -0,0 +1,271 @@
package agent
import (
"bytes"
"context"
"encoding/json"
"io"
"net/netip"
"sync"
"testing"
"time"
"github.com/stretchr/testify/require"
"google.golang.org/protobuf/types/known/durationpb"
"tailscale.com/types/ipproto"
"tailscale.com/types/netlogtype"
"cdr.dev/slog"
"cdr.dev/slog/sloggers/slogjson"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent/proto"
"github.com/coder/coder/v2/testutil"
)
func TestStatsReporter(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
logger := slogtest.Make(t, nil).Leveled(slog.LevelDebug)
fSource := newFakeNetworkStatsSource(ctx, t)
fCollector := newFakeCollector(t)
fDest := newFakeStatsDest()
uut := newStatsReporter(logger, fSource, fCollector)
loopErr := make(chan error, 1)
loopCtx, loopCancel := context.WithCancel(ctx)
go func() {
err := uut.reportLoop(loopCtx, fDest)
loopErr <- err
}()
// initial request to get duration
req := testutil.RequireRecvCtx(ctx, t, fDest.reqs)
require.NotNil(t, req)
require.Nil(t, req.Stats)
interval := time.Second * 34
testutil.RequireSendCtx(ctx, t, fDest.resps, &proto.UpdateStatsResponse{ReportInterval: durationpb.New(interval)})
// call to source to set the callback and interval
gotInterval := testutil.RequireRecvCtx(ctx, t, fSource.period)
require.Equal(t, interval, gotInterval)
// callback returning netstats
netStats := map[netlogtype.Connection]netlogtype.Counts{
{
Proto: ipproto.TCP,
Src: netip.MustParseAddrPort("192.168.1.33:4887"),
Dst: netip.MustParseAddrPort("192.168.2.99:9999"),
}: {
TxPackets: 22,
TxBytes: 23,
RxPackets: 24,
RxBytes: 25,
},
}
fSource.callback(time.Now(), time.Now(), netStats, nil)
// collector called to complete the stats
gotNetStats := testutil.RequireRecvCtx(ctx, t, fCollector.calls)
require.Equal(t, netStats, gotNetStats)
// while we are collecting the stats, send in two new netStats to simulate
// what happens if we don't keep up. Only the latest should be kept.
netStats0 := map[netlogtype.Connection]netlogtype.Counts{
{
Proto: ipproto.TCP,
Src: netip.MustParseAddrPort("192.168.1.33:4887"),
Dst: netip.MustParseAddrPort("192.168.2.99:9999"),
}: {
TxPackets: 10,
TxBytes: 10,
RxPackets: 10,
RxBytes: 10,
},
}
fSource.callback(time.Now(), time.Now(), netStats0, nil)
netStats1 := map[netlogtype.Connection]netlogtype.Counts{
{
Proto: ipproto.TCP,
Src: netip.MustParseAddrPort("192.168.1.33:4887"),
Dst: netip.MustParseAddrPort("192.168.2.99:9999"),
}: {
TxPackets: 11,
TxBytes: 11,
RxPackets: 11,
RxBytes: 11,
},
}
fSource.callback(time.Now(), time.Now(), netStats1, nil)
// complete first collection
stats := &proto.Stats{SessionCountJetbrains: 55}
testutil.RequireSendCtx(ctx, t, fCollector.stats, stats)
// destination called to report the first stats
update := testutil.RequireRecvCtx(ctx, t, fDest.reqs)
require.NotNil(t, update)
require.Equal(t, stats, update.Stats)
testutil.RequireSendCtx(ctx, t, fDest.resps, &proto.UpdateStatsResponse{ReportInterval: durationpb.New(interval)})
// second update -- only netStats1 is reported
gotNetStats = testutil.RequireRecvCtx(ctx, t, fCollector.calls)
require.Equal(t, netStats1, gotNetStats)
stats = &proto.Stats{SessionCountJetbrains: 66}
testutil.RequireSendCtx(ctx, t, fCollector.stats, stats)
update = testutil.RequireRecvCtx(ctx, t, fDest.reqs)
require.NotNil(t, update)
require.Equal(t, stats, update.Stats)
interval2 := 27 * time.Second
testutil.RequireSendCtx(ctx, t, fDest.resps, &proto.UpdateStatsResponse{ReportInterval: durationpb.New(interval2)})
// set the new interval
gotInterval = testutil.RequireRecvCtx(ctx, t, fSource.period)
require.Equal(t, interval2, gotInterval)
loopCancel()
err := testutil.RequireRecvCtx(ctx, t, loopErr)
require.NoError(t, err)
}
type fakeNetworkStatsSource struct {
sync.Mutex
ctx context.Context
t testing.TB
callback func(start, end time.Time, virtual, physical map[netlogtype.Connection]netlogtype.Counts)
period chan time.Duration
}
func (f *fakeNetworkStatsSource) SetConnStatsCallback(maxPeriod time.Duration, _ int, dump func(start time.Time, end time.Time, virtual map[netlogtype.Connection]netlogtype.Counts, physical map[netlogtype.Connection]netlogtype.Counts)) {
f.Lock()
defer f.Unlock()
f.callback = dump
select {
case <-f.ctx.Done():
f.t.Error("timeout")
case f.period <- maxPeriod:
// OK
}
}
func newFakeNetworkStatsSource(ctx context.Context, t testing.TB) *fakeNetworkStatsSource {
f := &fakeNetworkStatsSource{
ctx: ctx,
t: t,
period: make(chan time.Duration),
}
return f
}
type fakeCollector struct {
t testing.TB
calls chan map[netlogtype.Connection]netlogtype.Counts
stats chan *proto.Stats
}
func (f *fakeCollector) Collect(ctx context.Context, networkStats map[netlogtype.Connection]netlogtype.Counts) *proto.Stats {
select {
case <-ctx.Done():
f.t.Error("timeout on collect")
return nil
case f.calls <- networkStats:
// ok
}
select {
case <-ctx.Done():
f.t.Error("timeout on collect")
return nil
case s := <-f.stats:
return s
}
}
func newFakeCollector(t testing.TB) *fakeCollector {
return &fakeCollector{
t: t,
calls: make(chan map[netlogtype.Connection]netlogtype.Counts),
stats: make(chan *proto.Stats),
}
}
type fakeStatsDest struct {
reqs chan *proto.UpdateStatsRequest
resps chan *proto.UpdateStatsResponse
}
func (f *fakeStatsDest) UpdateStats(ctx context.Context, req *proto.UpdateStatsRequest) (*proto.UpdateStatsResponse, error) {
select {
case <-ctx.Done():
return nil, ctx.Err()
case f.reqs <- req:
// OK
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case resp := <-f.resps:
return resp, nil
}
}
func newFakeStatsDest() *fakeStatsDest {
return &fakeStatsDest{
reqs: make(chan *proto.UpdateStatsRequest),
resps: make(chan *proto.UpdateStatsResponse),
}
}
func Test_logDebouncer(t *testing.T) {
t.Parallel()
var (
buf bytes.Buffer
logger = slog.Make(slogjson.Sink(&buf))
ctx = context.Background()
)
debouncer := &logDebouncer{
logger: logger,
messages: map[string]time.Time{},
interval: time.Minute,
}
fields := map[string]interface{}{
"field_1": float64(1),
"field_2": "2",
}
debouncer.Error(ctx, "my message", "field_1", 1, "field_2", "2")
debouncer.Warn(ctx, "another message", "field_1", 1, "field_2", "2")
// Shouldn't log this.
debouncer.Warn(ctx, "another message", "field_1", 1, "field_2", "2")
require.Len(t, debouncer.messages, 2)
type entry struct {
Msg string `json:"msg"`
Level string `json:"level"`
Fields map[string]interface{} `json:"fields"`
}
assertLog := func(msg string, level string, fields map[string]interface{}) {
line, err := buf.ReadString('\n')
require.NoError(t, err)
var e entry
err = json.Unmarshal([]byte(line), &e)
require.NoError(t, err)
require.Equal(t, msg, e.Msg)
require.Equal(t, level, e.Level)
require.Equal(t, fields, e.Fields)
}
assertLog("my message", "ERROR", fields)
assertLog("another message", "WARN", fields)
debouncer.messages["another message"] = time.Now().Add(-2 * time.Minute)
debouncer.Warn(ctx, "another message", "field_1", 1, "field_2", "2")
assertLog("another message", "WARN", fields)
// Assert nothing else was written.
_, err := buf.ReadString('\n')
require.ErrorIs(t, err, io.EOF)
}
@@ -5,7 +5,7 @@ import (
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/coderd/util/apiversion"
"github.com/coder/coder/v2/apiversion"
)
func TestAPIVersionValidate(t *testing.T) {
+44 -54
View File
@@ -18,10 +18,8 @@ import (
"cloud.google.com/go/compute/metadata"
"golang.org/x/xerrors"
"gopkg.in/natefinch/lumberjack.v2"
"tailscale.com/util/clientmetric"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/common/expfmt"
"cdr.dev/slog"
"cdr.dev/slog/sloggers/sloghuman"
@@ -31,15 +29,16 @@ import (
"github.com/coder/coder/v2/agent/agentproc"
"github.com/coder/coder/v2/agent/reaper"
"github.com/coder/coder/v2/buildinfo"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/serpent"
)
func (r *RootCmd) workspaceAgent() *clibase.Cmd {
func (r *RootCmd) workspaceAgent() *serpent.Command {
var (
auth string
logDir string
scriptDataDir string
pprofAddress string
noReap bool
sshMaxTimeout time.Duration
@@ -50,12 +49,12 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
slogJSONPath string
slogStackdriverPath string
)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "agent",
Short: `Starts the Coder workspace agent.`,
// This command isn't useful to manually execute.
Hidden: true,
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
ctx, cancel := context.WithCancel(inv.Context())
defer cancel()
@@ -124,7 +123,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
args := append(os.Args, "--no-reap")
err := reaper.ForkReap(
reaper.WithExecArgs(args...),
reaper.WithCatchSignals(InterruptSignals...),
reaper.WithCatchSignals(StopSignals...),
)
if err != nil {
logger.Error(ctx, "agent process reaper unable to fork", slog.Error(err))
@@ -143,12 +142,12 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
// Note that we don't want to handle these signals in the
// process that runs as PID 1, that's why we do this after
// the reaper forked.
ctx, stopNotify := inv.SignalNotifyContext(ctx, InterruptSignals...)
ctx, stopNotify := inv.SignalNotifyContext(ctx, StopSignals...)
defer stopNotify()
// DumpHandler does signal handling, so we call it after the
// reaper.
go DumpHandler(ctx)
go DumpHandler(ctx, "agent")
logWriter := &lumberjackWriteCloseFixer{w: &lumberjack.Logger{
Filename: filepath.Join(logDir, "coder-agent.log"),
@@ -278,12 +277,21 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
subsystems = append(subsystems, subsystem)
}
procTicker := time.NewTicker(time.Second)
defer procTicker.Stop()
environmentVariables := map[string]string{
"GIT_ASKPASS": executablePath,
}
if v, ok := os.LookupEnv(agent.EnvProcPrioMgmt); ok {
environmentVariables[agent.EnvProcPrioMgmt] = v
}
if v, ok := os.LookupEnv(agent.EnvProcOOMScore); ok {
environmentVariables[agent.EnvProcOOMScore] = v
}
agnt := agent.New(agent.Options{
Client: client,
Logger: logger,
LogDir: logDir,
ScriptDataDir: scriptDataDir,
TailnetListenPort: uint16(tailnetListenPort),
ExchangeToken: func(ctx context.Context) (string, error) {
if exchangeToken == nil {
@@ -296,13 +304,10 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
client.SetSessionToken(resp.SessionToken)
return resp.SessionToken, nil
},
EnvironmentVariables: map[string]string{
"GIT_ASKPASS": executablePath,
agent.EnvProcPrioMgmt: os.Getenv(agent.EnvProcPrioMgmt),
},
IgnorePorts: ignorePorts,
SSHMaxTimeout: sshMaxTimeout,
Subsystems: subsystems,
EnvironmentVariables: environmentVariables,
IgnorePorts: ignorePorts,
SSHMaxTimeout: sshMaxTimeout,
Subsystems: subsystems,
PrometheusRegistry: prometheusRegistry,
Syscaller: agentproc.NewSyscaller(),
@@ -311,7 +316,8 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
ModifiedProcesses: nil,
})
prometheusSrvClose := ServeHandler(ctx, logger, prometheusMetricsHandler(prometheusRegistry, logger), prometheusAddress, "prometheus")
promHandler := agent.PrometheusMetricsHandler(prometheusRegistry, logger)
prometheusSrvClose := ServeHandler(ctx, logger, promHandler, prometheusAddress, "prometheus")
defer prometheusSrvClose()
debugSrvClose := ServeHandler(ctx, logger, agnt.HTTPDebug(), debugAddress, "debug")
@@ -322,26 +328,33 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
},
}
cmd.Options = clibase.OptionSet{
cmd.Options = serpent.OptionSet{
{
Flag: "auth",
Default: "token",
Description: "Specify the authentication type to use for the agent.",
Env: "CODER_AGENT_AUTH",
Value: clibase.StringOf(&auth),
Value: serpent.StringOf(&auth),
},
{
Flag: "log-dir",
Default: os.TempDir(),
Description: "Specify the location for the agent log files.",
Env: "CODER_AGENT_LOG_DIR",
Value: clibase.StringOf(&logDir),
Value: serpent.StringOf(&logDir),
},
{
Flag: "script-data-dir",
Default: os.TempDir(),
Description: "Specify the location for storing script data.",
Env: "CODER_AGENT_SCRIPT_DATA_DIR",
Value: serpent.StringOf(&scriptDataDir),
},
{
Flag: "pprof-address",
Default: "127.0.0.1:6060",
Env: "CODER_AGENT_PPROF_ADDRESS",
Value: clibase.StringOf(&pprofAddress),
Value: serpent.StringOf(&pprofAddress),
Description: "The address to serve pprof.",
},
{
@@ -349,7 +362,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Env: "",
Description: "Do not start a process reaper.",
Value: clibase.BoolOf(&noReap),
Value: serpent.BoolOf(&noReap),
},
{
Flag: "ssh-max-timeout",
@@ -357,27 +370,27 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Default: "72h",
Env: "CODER_AGENT_SSH_MAX_TIMEOUT",
Description: "Specify the max timeout for a SSH connection, it is advisable to set it to a minimum of 60s, but no more than 72h.",
Value: clibase.DurationOf(&sshMaxTimeout),
Value: serpent.DurationOf(&sshMaxTimeout),
},
{
Flag: "tailnet-listen-port",
Default: "0",
Env: "CODER_AGENT_TAILNET_LISTEN_PORT",
Description: "Specify a static port for Tailscale to use for listening.",
Value: clibase.Int64Of(&tailnetListenPort),
Value: serpent.Int64Of(&tailnetListenPort),
},
{
Flag: "prometheus-address",
Default: "127.0.0.1:2112",
Env: "CODER_AGENT_PROMETHEUS_ADDRESS",
Value: clibase.StringOf(&prometheusAddress),
Value: serpent.StringOf(&prometheusAddress),
Description: "The bind address to serve Prometheus metrics.",
},
{
Flag: "debug-address",
Default: "127.0.0.1:2113",
Env: "CODER_AGENT_DEBUG_ADDRESS",
Value: clibase.StringOf(&debugAddress),
Value: serpent.StringOf(&debugAddress),
Description: "The bind address to serve a debug HTTP server.",
},
{
@@ -386,7 +399,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Flag: "log-human",
Env: "CODER_AGENT_LOGGING_HUMAN",
Default: "/dev/stderr",
Value: clibase.StringOf(&slogHumanPath),
Value: serpent.StringOf(&slogHumanPath),
},
{
Name: "JSON Log Location",
@@ -394,7 +407,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Flag: "log-json",
Env: "CODER_AGENT_LOGGING_JSON",
Default: "",
Value: clibase.StringOf(&slogJSONPath),
Value: serpent.StringOf(&slogJSONPath),
},
{
Name: "Stackdriver Log Location",
@@ -402,7 +415,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Flag: "log-stackdriver",
Env: "CODER_AGENT_LOGGING_STACKDRIVER",
Default: "",
Value: clibase.StringOf(&slogStackdriverPath),
Value: serpent.StringOf(&slogStackdriverPath),
},
}
@@ -490,26 +503,3 @@ func urlPort(u string) (int, error) {
}
return -1, xerrors.Errorf("invalid port: %s", u)
}
func prometheusMetricsHandler(prometheusRegistry *prometheus.Registry, logger slog.Logger) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
// Based on: https://github.com/tailscale/tailscale/blob/280255acae604796a1113861f5a84e6fa2dc6121/ipn/localapi/localapi.go#L489
clientmetric.WritePrometheusExpositionFormat(w)
metricFamilies, err := prometheusRegistry.Gather()
if err != nil {
logger.Error(context.Background(), "Prometheus handler can't gather metric families", slog.Error(err))
return
}
for _, metricFamily := range metricFamilies {
_, err = expfmt.MetricFamilyToText(w, metricFamily)
if err != nil {
logger.Error(context.Background(), "expfmt.MetricFamilyToText failed", slog.Error(err))
return
}
}
})
}
+42 -7
View File
@@ -19,6 +19,7 @@ import (
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/dbfake"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
"github.com/coder/coder/v2/provisionersdk/proto"
"github.com/coder/coder/v2/testutil"
)
@@ -83,14 +84,16 @@ func TestWorkspaceAgent(t *testing.T) {
ctx := inv.Context()
clitest.Start(t, inv)
coderdtest.AwaitWorkspaceAgents(t, client, r.Workspace.ID)
coderdtest.NewWorkspaceAgentWaiter(t, client, r.Workspace.ID).
MatchResources(matchAgentWithVersion).Wait()
workspace, err := client.Workspace(ctx, r.Workspace.ID)
require.NoError(t, err)
resources := workspace.LatestBuild.Resources
if assert.NotEmpty(t, workspace.LatestBuild.Resources) && assert.NotEmpty(t, resources[0].Agents) {
assert.NotEmpty(t, resources[0].Agents[0].Version)
}
dialer, err := client.DialWorkspaceAgent(ctx, resources[0].Agents[0].ID, nil)
dialer, err := workspacesdk.New(client).
DialAgent(ctx, resources[0].Agents[0].ID, nil)
require.NoError(t, err)
defer dialer.Close()
require.True(t, dialer.AwaitReachable(ctx))
@@ -120,14 +123,17 @@ func TestWorkspaceAgent(t *testing.T) {
clitest.Start(t, inv)
ctx := inv.Context()
coderdtest.AwaitWorkspaceAgents(t, client, r.Workspace.ID)
coderdtest.NewWorkspaceAgentWaiter(t, client, r.Workspace.ID).
MatchResources(matchAgentWithVersion).
Wait()
workspace, err := client.Workspace(ctx, r.Workspace.ID)
require.NoError(t, err)
resources := workspace.LatestBuild.Resources
if assert.NotEmpty(t, resources) && assert.NotEmpty(t, resources[0].Agents) {
assert.NotEmpty(t, resources[0].Agents[0].Version)
}
dialer, err := client.DialWorkspaceAgent(ctx, resources[0].Agents[0].ID, nil)
dialer, err := workspacesdk.New(client).
DialAgent(ctx, resources[0].Agents[0].ID, nil)
require.NoError(t, err)
defer dialer.Close()
require.True(t, dialer.AwaitReachable(ctx))
@@ -161,14 +167,16 @@ func TestWorkspaceAgent(t *testing.T) {
)
ctx := inv.Context()
coderdtest.AwaitWorkspaceAgents(t, client, r.Workspace.ID)
coderdtest.NewWorkspaceAgentWaiter(t, client, r.Workspace.ID).
MatchResources(matchAgentWithVersion).
Wait()
workspace, err := client.Workspace(ctx, r.Workspace.ID)
require.NoError(t, err)
resources := workspace.LatestBuild.Resources
if assert.NotEmpty(t, resources) && assert.NotEmpty(t, resources[0].Agents) {
assert.NotEmpty(t, resources[0].Agents[0].Version)
}
dialer, err := client.DialWorkspaceAgent(ctx, resources[0].Agents[0].ID, nil)
dialer, err := workspacesdk.New(client).DialAgent(ctx, resources[0].Agents[0].ID, nil)
require.NoError(t, err)
defer dialer.Close()
require.True(t, dialer.AwaitReachable(ctx))
@@ -212,7 +220,8 @@ func TestWorkspaceAgent(t *testing.T) {
clitest.Start(t, inv)
resources := coderdtest.AwaitWorkspaceAgents(t, client, r.Workspace.ID)
resources := coderdtest.NewWorkspaceAgentWaiter(t, client, r.Workspace.ID).
MatchResources(matchAgentWithSubsystems).Wait()
require.Len(t, resources, 1)
require.Len(t, resources[0].Agents, 1)
require.Len(t, resources[0].Agents[0].Subsystems, 2)
@@ -221,3 +230,29 @@ func TestWorkspaceAgent(t *testing.T) {
require.Equal(t, codersdk.AgentSubsystemExectrace, resources[0].Agents[0].Subsystems[1])
})
}
func matchAgentWithVersion(rs []codersdk.WorkspaceResource) bool {
if len(rs) < 1 {
return false
}
if len(rs[0].Agents) < 1 {
return false
}
if rs[0].Agents[0].Version == "" {
return false
}
return true
}
func matchAgentWithSubsystems(rs []codersdk.WorkspaceResource) bool {
if len(rs) < 1 {
return false
}
if len(rs[0].Agents) < 1 {
return false
}
if len(rs[0].Agents[0].Subsystems) < 1 {
return false
}
return true
}
+6 -6
View File
@@ -6,22 +6,22 @@ import (
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
func (r *RootCmd) autoupdate() *clibase.Cmd {
func (r *RootCmd) autoupdate() *serpent.Command {
client := new(codersdk.Client)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Annotations: workspaceCommand,
Use: "autoupdate <workspace> <always|never>",
Short: "Toggle auto-update policy for a workspace",
Middleware: clibase.Chain(
clibase.RequireNArgs(2),
Middleware: serpent.Chain(
serpent.RequireNArgs(2),
r.InitClient(client),
),
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
policy := strings.ToLower(inv.Args[1])
err := validateAutoUpdatePolicy(policy)
if err != nil {
-80
View File
@@ -1,80 +0,0 @@
// Package clibase offers an all-in-one solution for a highly configurable CLI
// application. Within Coder, we use it for all of our subcommands, which
// demands more functionality than cobra/viber offers.
//
// The Command interface is loosely based on the chi middleware pattern and
// http.Handler/HandlerFunc.
package clibase
import (
"strings"
"golang.org/x/exp/maps"
)
// Group describes a hierarchy of groups that an option or command belongs to.
type Group struct {
Parent *Group `json:"parent,omitempty"`
Name string `json:"name,omitempty"`
YAML string `json:"yaml,omitempty"`
Description string `json:"description,omitempty"`
}
// Ancestry returns the group and all of its parents, in order.
func (g *Group) Ancestry() []Group {
if g == nil {
return nil
}
groups := []Group{*g}
for p := g.Parent; p != nil; p = p.Parent {
// Prepend to the slice so that the order is correct.
groups = append([]Group{*p}, groups...)
}
return groups
}
func (g *Group) FullName() string {
var names []string
for _, g := range g.Ancestry() {
names = append(names, g.Name)
}
return strings.Join(names, " / ")
}
// Annotations is an arbitrary key-mapping used to extend the Option and Command types.
// Its methods won't panic if the map is nil.
type Annotations map[string]string
// Mark sets a value on the annotations map, creating one
// if it doesn't exist. Mark does not mutate the original and
// returns a copy. It is suitable for chaining.
func (a Annotations) Mark(key string, value string) Annotations {
var aa Annotations
if a != nil {
aa = maps.Clone(a)
} else {
aa = make(Annotations)
}
aa[key] = value
return aa
}
// IsSet returns true if the key is set in the annotations map.
func (a Annotations) IsSet(key string) bool {
if a == nil {
return false
}
_, ok := a[key]
return ok
}
// Get retrieves a key from the map, returning false if the key is not found
// or the map is nil.
func (a Annotations) Get(key string) (string, bool) {
if a == nil {
return "", false
}
v, ok := a[key]
return v, ok
}
-621
View File
@@ -1,621 +0,0 @@
package clibase
import (
"context"
"errors"
"flag"
"fmt"
"io"
"os"
"os/signal"
"strings"
"testing"
"unicode"
"cdr.dev/slog"
"github.com/spf13/pflag"
"golang.org/x/exp/slices"
"golang.org/x/xerrors"
"gopkg.in/yaml.v3"
"github.com/coder/coder/v2/coderd/util/slice"
)
// Cmd describes an executable command.
type Cmd struct {
// Parent is the direct parent of the command.
Parent *Cmd
// Children is a list of direct descendants.
Children []*Cmd
// Use is provided in form "command [flags] [args...]".
Use string
// Aliases is a list of alternative names for the command.
Aliases []string
// Short is a one-line description of the command.
Short string
// Hidden determines whether the command should be hidden from help.
Hidden bool
// RawArgs determines whether the command should receive unparsed arguments.
// No flags are parsed when set, and the command is responsible for parsing
// its own flags.
RawArgs bool
// Long is a detailed description of the command,
// presented on its help page. It may contain examples.
Long string
Options OptionSet
Annotations Annotations
// Middleware is called before the Handler.
// Use Chain() to combine multiple middlewares.
Middleware MiddlewareFunc
Handler HandlerFunc
HelpHandler HandlerFunc
}
// AddSubcommands adds the given subcommands, setting their
// Parent field automatically.
func (c *Cmd) AddSubcommands(cmds ...*Cmd) {
for _, cmd := range cmds {
cmd.Parent = c
c.Children = append(c.Children, cmd)
}
}
// Walk calls fn for the command and all its children.
func (c *Cmd) Walk(fn func(*Cmd)) {
fn(c)
for _, child := range c.Children {
child.Parent = c
child.Walk(fn)
}
}
// PrepareAll performs initialization and linting on the command and all its children.
func (c *Cmd) PrepareAll() error {
if c.Use == "" {
return xerrors.New("command must have a Use field so that it has a name")
}
var merr error
for i := range c.Options {
opt := &c.Options[i]
if opt.Name == "" {
switch {
case opt.Flag != "":
opt.Name = opt.Flag
case opt.Env != "":
opt.Name = opt.Env
case opt.YAML != "":
opt.Name = opt.YAML
default:
merr = errors.Join(merr, xerrors.Errorf("option must have a Name, Flag, Env or YAML field"))
}
}
if opt.Description != "" {
// Enforce that description uses sentence form.
if unicode.IsLower(rune(opt.Description[0])) {
merr = errors.Join(merr, xerrors.Errorf("option %q description should start with a capital letter", opt.Name))
}
if !strings.HasSuffix(opt.Description, ".") {
merr = errors.Join(merr, xerrors.Errorf("option %q description should end with a period", opt.Name))
}
}
}
slices.SortFunc(c.Options, func(a, b Option) int {
return slice.Ascending(a.Name, b.Name)
})
slices.SortFunc(c.Children, func(a, b *Cmd) int {
return slice.Ascending(a.Name(), b.Name())
})
for _, child := range c.Children {
child.Parent = c
err := child.PrepareAll()
if err != nil {
merr = errors.Join(merr, xerrors.Errorf("command %v: %w", child.Name(), err))
}
}
return merr
}
// Name returns the first word in the Use string.
func (c *Cmd) Name() string {
return strings.Split(c.Use, " ")[0]
}
// FullName returns the full invocation name of the command,
// as seen on the command line.
func (c *Cmd) FullName() string {
var names []string
if c.Parent != nil {
names = append(names, c.Parent.FullName())
}
names = append(names, c.Name())
return strings.Join(names, " ")
}
// FullName returns usage of the command, preceded
// by the usage of its parents.
func (c *Cmd) FullUsage() string {
var uses []string
if c.Parent != nil {
uses = append(uses, c.Parent.FullName())
}
uses = append(uses, c.Use)
return strings.Join(uses, " ")
}
// FullOptions returns the options of the command and its parents.
func (c *Cmd) FullOptions() OptionSet {
var opts OptionSet
if c.Parent != nil {
opts = append(opts, c.Parent.FullOptions()...)
}
opts = append(opts, c.Options...)
return opts
}
// Invoke creates a new invocation of the command, with
// stdio discarded.
//
// The returned invocation is not live until Run() is called.
func (c *Cmd) Invoke(args ...string) *Invocation {
return &Invocation{
Command: c,
Args: args,
Stdout: io.Discard,
Stderr: io.Discard,
Stdin: strings.NewReader(""),
Logger: slog.Make(),
}
}
// Invocation represents an instance of a command being executed.
type Invocation struct {
ctx context.Context
Command *Cmd
parsedFlags *pflag.FlagSet
Args []string
// Environ is a list of environment variables. Use EnvsWithPrefix to parse
// os.Environ.
Environ Environ
Stdout io.Writer
Stderr io.Writer
Stdin io.Reader
Logger slog.Logger
Net Net
// testing
signalNotifyContext func(parent context.Context, signals ...os.Signal) (ctx context.Context, stop context.CancelFunc)
}
// WithOS returns the invocation as a main package, filling in the invocation's unset
// fields with OS defaults.
func (inv *Invocation) WithOS() *Invocation {
return inv.with(func(i *Invocation) {
i.Stdout = os.Stdout
i.Stderr = os.Stderr
i.Stdin = os.Stdin
i.Args = os.Args[1:]
i.Environ = ParseEnviron(os.Environ(), "")
i.Net = osNet{}
})
}
// WithTestSignalNotifyContext allows overriding the default implementation of SignalNotifyContext.
// This should only be used in testing.
func (inv *Invocation) WithTestSignalNotifyContext(
_ testing.TB, // ensure we only call this from tests
f func(parent context.Context, signals ...os.Signal) (ctx context.Context, stop context.CancelFunc),
) *Invocation {
return inv.with(func(i *Invocation) {
i.signalNotifyContext = f
})
}
// SignalNotifyContext is equivalent to signal.NotifyContext, but supports being overridden in
// tests.
func (inv *Invocation) SignalNotifyContext(parent context.Context, signals ...os.Signal) (ctx context.Context, stop context.CancelFunc) {
if inv.signalNotifyContext == nil {
return signal.NotifyContext(parent, signals...)
}
return inv.signalNotifyContext(parent, signals...)
}
func (inv *Invocation) WithTestParsedFlags(
_ testing.TB, // ensure we only call this from tests
parsedFlags *pflag.FlagSet,
) *Invocation {
return inv.with(func(i *Invocation) {
i.parsedFlags = parsedFlags
})
}
func (inv *Invocation) Context() context.Context {
if inv.ctx == nil {
return context.Background()
}
return inv.ctx
}
func (inv *Invocation) ParsedFlags() *pflag.FlagSet {
if inv.parsedFlags == nil {
panic("flags not parsed, has Run() been called?")
}
return inv.parsedFlags
}
type runState struct {
allArgs []string
commandDepth int
flagParseErr error
}
func copyFlagSetWithout(fs *pflag.FlagSet, without string) *pflag.FlagSet {
fs2 := pflag.NewFlagSet("", pflag.ContinueOnError)
fs2.Usage = func() {}
fs.VisitAll(func(f *pflag.Flag) {
if f.Name == without {
return
}
fs2.AddFlag(f)
})
return fs2
}
// run recursively executes the command and its children.
// allArgs is wired through the stack so that global flags can be accepted
// anywhere in the command invocation.
func (inv *Invocation) run(state *runState) error {
err := inv.Command.Options.ParseEnv(inv.Environ)
if err != nil {
return xerrors.Errorf("parsing env: %w", err)
}
// Now the fun part, argument parsing!
children := make(map[string]*Cmd)
for _, child := range inv.Command.Children {
child.Parent = inv.Command
for _, name := range append(child.Aliases, child.Name()) {
if _, ok := children[name]; ok {
return xerrors.Errorf("duplicate command name: %s", name)
}
children[name] = child
}
}
if inv.parsedFlags == nil {
inv.parsedFlags = pflag.NewFlagSet(inv.Command.Name(), pflag.ContinueOnError)
// We handle Usage ourselves.
inv.parsedFlags.Usage = func() {}
}
// If we find a duplicate flag, we want the deeper command's flag to override
// the shallow one. Unfortunately, pflag has no way to remove a flag, so we
// have to create a copy of the flagset without a value.
inv.Command.Options.FlagSet().VisitAll(func(f *pflag.Flag) {
if inv.parsedFlags.Lookup(f.Name) != nil {
inv.parsedFlags = copyFlagSetWithout(inv.parsedFlags, f.Name)
}
inv.parsedFlags.AddFlag(f)
})
var parsedArgs []string
if !inv.Command.RawArgs {
// Flag parsing will fail on intermediate commands in the command tree,
// so we check the error after looking for a child command.
state.flagParseErr = inv.parsedFlags.Parse(state.allArgs)
parsedArgs = inv.parsedFlags.Args()
}
// Set value sources for flags.
for i, opt := range inv.Command.Options {
if fl := inv.parsedFlags.Lookup(opt.Flag); fl != nil && fl.Changed {
inv.Command.Options[i].ValueSource = ValueSourceFlag
}
}
// Read YAML configs, if any.
for _, opt := range inv.Command.Options {
path, ok := opt.Value.(*YAMLConfigPath)
if !ok || path.String() == "" {
continue
}
byt, err := os.ReadFile(path.String())
if err != nil {
return xerrors.Errorf("reading yaml: %w", err)
}
var n yaml.Node
err = yaml.Unmarshal(byt, &n)
if err != nil {
return xerrors.Errorf("decoding yaml: %w", err)
}
err = inv.Command.Options.UnmarshalYAML(&n)
if err != nil {
return xerrors.Errorf("applying yaml: %w", err)
}
}
err = inv.Command.Options.SetDefaults()
if err != nil {
return xerrors.Errorf("setting defaults: %w", err)
}
// Run child command if found (next child only)
// We must do subcommand detection after flag parsing so we don't mistake flag
// values for subcommand names.
if len(parsedArgs) > state.commandDepth {
nextArg := parsedArgs[state.commandDepth]
if child, ok := children[nextArg]; ok {
child.Parent = inv.Command
inv.Command = child
state.commandDepth++
return inv.run(state)
}
}
// Flag parse errors are irrelevant for raw args commands.
if !inv.Command.RawArgs && state.flagParseErr != nil && !errors.Is(state.flagParseErr, pflag.ErrHelp) {
return xerrors.Errorf(
"parsing flags (%v) for %q: %w",
state.allArgs,
inv.Command.FullName(), state.flagParseErr,
)
}
// All options should be set. Check all required options have sources,
// meaning they were set by the user in some way (env, flag, etc).
var missing []string
for _, opt := range inv.Command.Options {
if opt.Required && opt.ValueSource == ValueSourceNone {
missing = append(missing, opt.Flag)
}
}
if len(missing) > 0 {
return xerrors.Errorf("Missing values for the required flags: %s", strings.Join(missing, ", "))
}
if inv.Command.RawArgs {
// If we're at the root command, then the name is omitted
// from the arguments, so we can just use the entire slice.
if state.commandDepth == 0 {
inv.Args = state.allArgs
} else {
argPos, err := findArg(inv.Command.Name(), state.allArgs, inv.parsedFlags)
if err != nil {
panic(err)
}
inv.Args = state.allArgs[argPos+1:]
}
} else {
// In non-raw-arg mode, we want to skip over flags.
inv.Args = parsedArgs[state.commandDepth:]
}
mw := inv.Command.Middleware
if mw == nil {
mw = Chain()
}
ctx := inv.ctx
if ctx == nil {
ctx = context.Background()
}
ctx, cancel := context.WithCancel(ctx)
defer cancel()
inv = inv.WithContext(ctx)
if inv.Command.Handler == nil || errors.Is(state.flagParseErr, pflag.ErrHelp) {
if inv.Command.HelpHandler == nil {
return xerrors.Errorf("no handler or help for command %s", inv.Command.FullName())
}
return inv.Command.HelpHandler(inv)
}
err = mw(inv.Command.Handler)(inv)
if err != nil {
return &RunCommandError{
Cmd: inv.Command,
Err: err,
}
}
return nil
}
type RunCommandError struct {
Cmd *Cmd
Err error
}
func (e *RunCommandError) Unwrap() error {
return e.Err
}
func (e *RunCommandError) Error() string {
return fmt.Sprintf("running command %q: %+v", e.Cmd.FullName(), e.Err)
}
// findArg returns the index of the first occurrence of arg in args, skipping
// over all flags.
func findArg(want string, args []string, fs *pflag.FlagSet) (int, error) {
for i := 0; i < len(args); i++ {
arg := args[i]
if !strings.HasPrefix(arg, "-") {
if arg == want {
return i, nil
}
continue
}
// This is a flag!
if strings.Contains(arg, "=") {
// The flag contains the value in the same arg, just skip.
continue
}
// We need to check if NoOptValue is set, then we should not wait
// for the next arg to be the value.
f := fs.Lookup(strings.TrimLeft(arg, "-"))
if f == nil {
return -1, xerrors.Errorf("unknown flag: %s", arg)
}
if f.NoOptDefVal != "" {
continue
}
if i == len(args)-1 {
return -1, xerrors.Errorf("flag %s requires a value", arg)
}
// Skip the value.
i++
}
return -1, xerrors.Errorf("arg %s not found", want)
}
// Run executes the command.
// If two command share a flag name, the first command wins.
//
//nolint:revive
func (inv *Invocation) Run() (err error) {
defer func() {
// Pflag is panicky, so additional context is helpful in tests.
if flag.Lookup("test.v") == nil {
return
}
if r := recover(); r != nil {
err = xerrors.Errorf("panic recovered for %s: %v", inv.Command.FullName(), r)
panic(err)
}
}()
// We close Stdin to prevent deadlocks, e.g. when the command
// has ended but an io.Copy is still reading from Stdin.
defer func() {
if inv.Stdin == nil {
return
}
rc, ok := inv.Stdin.(io.ReadCloser)
if !ok {
return
}
e := rc.Close()
err = errors.Join(err, e)
}()
err = inv.run(&runState{
allArgs: inv.Args,
})
return err
}
// WithContext returns a copy of the Invocation with the given context.
func (inv *Invocation) WithContext(ctx context.Context) *Invocation {
return inv.with(func(i *Invocation) {
i.ctx = ctx
})
}
// with returns a copy of the Invocation with the given function applied.
func (inv *Invocation) with(fn func(*Invocation)) *Invocation {
i2 := *inv
fn(&i2)
return &i2
}
// MiddlewareFunc returns the next handler in the chain,
// or nil if there are no more.
type MiddlewareFunc func(next HandlerFunc) HandlerFunc
func chain(ms ...MiddlewareFunc) MiddlewareFunc {
return MiddlewareFunc(func(next HandlerFunc) HandlerFunc {
if len(ms) > 0 {
return chain(ms[1:]...)(ms[0](next))
}
return next
})
}
// Chain returns a Handler that first calls middleware in order.
//
//nolint:revive
func Chain(ms ...MiddlewareFunc) MiddlewareFunc {
// We need to reverse the array to provide top-to-bottom execution
// order when defining a command.
reversed := make([]MiddlewareFunc, len(ms))
for i := range ms {
reversed[len(ms)-1-i] = ms[i]
}
return chain(reversed...)
}
func RequireNArgs(want int) MiddlewareFunc {
return RequireRangeArgs(want, want)
}
// RequireRangeArgs returns a Middleware that requires the number of arguments
// to be between start and end (inclusive). If end is -1, then the number of
// arguments must be at least start.
func RequireRangeArgs(start, end int) MiddlewareFunc {
if start < 0 {
panic("start must be >= 0")
}
return func(next HandlerFunc) HandlerFunc {
return func(i *Invocation) error {
got := len(i.Args)
switch {
case start == end && got != start:
switch start {
case 0:
if len(i.Command.Children) > 0 {
return xerrors.Errorf("unrecognized subcommand %q", i.Args[0])
}
return xerrors.Errorf("wanted no args but got %v %v", got, i.Args)
default:
return xerrors.Errorf(
"wanted %v args but got %v %v",
start,
got,
i.Args,
)
}
case start > 0 && end == -1:
switch {
case got < start:
return xerrors.Errorf(
"wanted at least %v args but got %v",
start,
got,
)
default:
return next(i)
}
case start > end:
panic("start must be <= end")
case got < start || got > end:
return xerrors.Errorf(
"wanted between %v and %v args but got %v",
start, end,
got,
)
default:
return next(i)
}
}
}
}
// HandlerFunc handles an Invocation of a command.
type HandlerFunc func(i *Invocation) error
-719
View File
@@ -1,719 +0,0 @@
package clibase_test
import (
"bytes"
"context"
"fmt"
"os"
"strings"
"testing"
"github.com/stretchr/testify/require"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
)
// ioBufs is the standard input, output, and error for a command.
type ioBufs struct {
Stdin bytes.Buffer
Stdout bytes.Buffer
Stderr bytes.Buffer
}
// fakeIO sets Stdin, Stdout, and Stderr to buffers.
func fakeIO(i *clibase.Invocation) *ioBufs {
var b ioBufs
i.Stdout = &b.Stdout
i.Stderr = &b.Stderr
i.Stdin = &b.Stdin
return &b
}
func TestCommand(t *testing.T) {
t.Parallel()
cmd := func() *clibase.Cmd {
var (
verbose bool
lower bool
prefix string
reqBool bool
reqStr string
)
return &clibase.Cmd{
Use: "root [subcommand]",
Options: clibase.OptionSet{
clibase.Option{
Name: "verbose",
Flag: "verbose",
Value: clibase.BoolOf(&verbose),
},
clibase.Option{
Name: "prefix",
Flag: "prefix",
Value: clibase.StringOf(&prefix),
},
},
Children: []*clibase.Cmd{
{
Use: "required-flag --req-bool=true --req-string=foo",
Short: "Example with required flags",
Options: clibase.OptionSet{
clibase.Option{
Name: "req-bool",
Flag: "req-bool",
Value: clibase.BoolOf(&reqBool),
Required: true,
},
clibase.Option{
Name: "req-string",
Flag: "req-string",
Value: clibase.Validate(clibase.StringOf(&reqStr), func(value *clibase.String) error {
ok := strings.Contains(value.String(), " ")
if !ok {
return xerrors.Errorf("string must contain a space")
}
return nil
}),
Required: true,
},
},
Handler: func(i *clibase.Invocation) error {
_, _ = i.Stdout.Write([]byte(fmt.Sprintf("%s-%t", reqStr, reqBool)))
return nil
},
},
{
Use: "toupper [word]",
Short: "Converts a word to upper case",
Middleware: clibase.Chain(
clibase.RequireNArgs(1),
),
Aliases: []string{"up"},
Options: clibase.OptionSet{
clibase.Option{
Name: "lower",
Flag: "lower",
Value: clibase.BoolOf(&lower),
},
},
Handler: func(i *clibase.Invocation) error {
_, _ = i.Stdout.Write([]byte(prefix))
w := i.Args[0]
if lower {
w = strings.ToLower(w)
} else {
w = strings.ToUpper(w)
}
_, _ = i.Stdout.Write(
[]byte(
w,
),
)
if verbose {
i.Stdout.Write([]byte("!!!"))
}
return nil
},
},
},
}
}
t.Run("SimpleOK", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke("toupper", "hello")
io := fakeIO(i)
i.Run()
require.Equal(t, "HELLO", io.Stdout.String())
})
t.Run("Alias", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"up", "hello",
)
io := fakeIO(i)
i.Run()
require.Equal(t, "HELLO", io.Stdout.String())
})
t.Run("NoSubcommand", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"na",
)
io := fakeIO(i)
err := i.Run()
require.Empty(t, io.Stdout.String())
require.Error(t, err)
})
t.Run("BadArgs", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper",
)
io := fakeIO(i)
err := i.Run()
require.Empty(t, io.Stdout.String())
require.Error(t, err)
})
t.Run("UnknownFlags", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper", "--unknown",
)
io := fakeIO(i)
err := i.Run()
require.Empty(t, io.Stdout.String())
require.Error(t, err)
})
t.Run("Verbose", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"--verbose", "toupper", "hello",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "HELLO!!!", io.Stdout.String())
})
t.Run("Verbose=", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"--verbose=true", "toupper", "hello",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "HELLO!!!", io.Stdout.String())
})
t.Run("PrefixSpace", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"--prefix", "conv: ", "toupper", "hello",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "conv: HELLO", io.Stdout.String())
})
t.Run("GlobalFlagsAnywhere", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper", "--prefix", "conv: ", "hello", "--verbose",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "conv: HELLO!!!", io.Stdout.String())
})
t.Run("LowerVerbose", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper", "--verbose", "hello", "--lower",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "hello!!!", io.Stdout.String())
})
t.Run("ParsedFlags", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper", "--verbose", "hello", "--lower",
)
_ = fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t,
"true",
i.ParsedFlags().Lookup("verbose").Value.String(),
)
})
t.Run("NoDeepChild", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"root", "level", "level", "toupper", "--verbose", "hello", "--lower",
)
fio := fakeIO(i)
require.Error(t, i.Run(), fio.Stdout.String())
})
t.Run("RequiredFlagsMissing", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag",
)
fio := fakeIO(i)
err := i.Run()
require.Error(t, err, fio.Stdout.String())
require.ErrorContains(t, err, "Missing values")
})
t.Run("RequiredFlagsMissingBool", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag", "--req-string", "foo bar",
)
fio := fakeIO(i)
err := i.Run()
require.Error(t, err, fio.Stdout.String())
require.ErrorContains(t, err, "Missing values for the required flags: req-bool")
})
t.Run("RequiredFlagsMissingString", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag", "--req-bool", "true",
)
fio := fakeIO(i)
err := i.Run()
require.Error(t, err, fio.Stdout.String())
require.ErrorContains(t, err, "Missing values for the required flags: req-string")
})
t.Run("RequiredFlagsInvalid", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag", "--req-string", "nospace",
)
fio := fakeIO(i)
err := i.Run()
require.Error(t, err, fio.Stdout.String())
require.ErrorContains(t, err, "string must contain a space")
})
t.Run("RequiredFlagsOK", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag", "--req-bool", "true", "--req-string", "foo bar",
)
fio := fakeIO(i)
err := i.Run()
require.NoError(t, err, fio.Stdout.String())
})
}
func TestCommand_DeepNest(t *testing.T) {
t.Parallel()
cmd := &clibase.Cmd{
Use: "1",
Children: []*clibase.Cmd{
{
Use: "2",
Children: []*clibase.Cmd{
{
Use: "3",
Handler: func(i *clibase.Invocation) error {
i.Stdout.Write([]byte("3"))
return nil
},
},
},
},
},
}
inv := cmd.Invoke("2", "3")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Equal(t, "3", stdio.Stdout.String())
}
func TestCommand_FlagOverride(t *testing.T) {
t.Parallel()
var flag string
cmd := &clibase.Cmd{
Use: "1",
Options: clibase.OptionSet{
{
Name: "flag",
Flag: "f",
Value: clibase.DiscardValue,
},
},
Children: []*clibase.Cmd{
{
Use: "2",
Options: clibase.OptionSet{
{
Name: "flag",
Flag: "f",
Value: clibase.StringOf(&flag),
},
},
Handler: func(i *clibase.Invocation) error {
return nil
},
},
},
}
err := cmd.Invoke("2", "--f", "mhmm").Run()
require.NoError(t, err)
require.Equal(t, "mhmm", flag)
}
func TestCommand_MiddlewareOrder(t *testing.T) {
t.Parallel()
mw := func(letter string) clibase.MiddlewareFunc {
return func(next clibase.HandlerFunc) clibase.HandlerFunc {
return (func(i *clibase.Invocation) error {
_, _ = i.Stdout.Write([]byte(letter))
return next(i)
})
}
}
cmd := &clibase.Cmd{
Use: "toupper [word]",
Short: "Converts a word to upper case",
Middleware: clibase.Chain(
mw("A"),
mw("B"),
mw("C"),
),
Handler: (func(i *clibase.Invocation) error {
return nil
}),
}
i := cmd.Invoke(
"hello", "world",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "ABC", io.Stdout.String())
}
func TestCommand_RawArgs(t *testing.T) {
t.Parallel()
cmd := func() *clibase.Cmd {
return &clibase.Cmd{
Use: "root",
Options: clibase.OptionSet{
{
Name: "password",
Flag: "password",
Value: clibase.StringOf(new(string)),
},
},
Children: []*clibase.Cmd{
{
Use: "sushi <args...>",
Short: "Throws back raw output",
RawArgs: true,
Handler: (func(i *clibase.Invocation) error {
if v := i.ParsedFlags().Lookup("password").Value.String(); v != "codershack" {
return xerrors.Errorf("password %q is wrong!", v)
}
i.Stdout.Write([]byte(strings.Join(i.Args, " ")))
return nil
}),
},
},
}
}
t.Run("OK", func(t *testing.T) {
// Flag parsed before the raw arg command should still work.
t.Parallel()
i := cmd().Invoke(
"--password", "codershack", "sushi", "hello", "--verbose", "world",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "hello --verbose world", io.Stdout.String())
})
t.Run("BadFlag", func(t *testing.T) {
// Verbose before the raw arg command should fail.
t.Parallel()
i := cmd().Invoke(
"--password", "codershack", "--verbose", "sushi", "hello", "world",
)
io := fakeIO(i)
require.Error(t, i.Run())
require.Empty(t, io.Stdout.String())
})
t.Run("NoPassword", func(t *testing.T) {
// Flag parsed before the raw arg command should still work.
t.Parallel()
i := cmd().Invoke(
"sushi", "hello", "--verbose", "world",
)
_ = fakeIO(i)
require.Error(t, i.Run())
})
}
func TestCommand_RootRaw(t *testing.T) {
t.Parallel()
cmd := &clibase.Cmd{
RawArgs: true,
Handler: func(i *clibase.Invocation) error {
i.Stdout.Write([]byte(strings.Join(i.Args, " ")))
return nil
},
}
inv := cmd.Invoke("hello", "--verbose", "--friendly")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Equal(t, "hello --verbose --friendly", stdio.Stdout.String())
}
func TestCommand_HyphenHyphen(t *testing.T) {
t.Parallel()
cmd := &clibase.Cmd{
Handler: (func(i *clibase.Invocation) error {
i.Stdout.Write([]byte(strings.Join(i.Args, " ")))
return nil
}),
}
inv := cmd.Invoke("--", "--verbose", "--friendly")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Equal(t, "--verbose --friendly", stdio.Stdout.String())
}
func TestCommand_ContextCancels(t *testing.T) {
t.Parallel()
var gotCtx context.Context
cmd := &clibase.Cmd{
Handler: (func(i *clibase.Invocation) error {
gotCtx = i.Context()
if err := gotCtx.Err(); err != nil {
return xerrors.Errorf("unexpected context error: %w", i.Context().Err())
}
return nil
}),
}
err := cmd.Invoke().Run()
require.NoError(t, err)
require.Error(t, gotCtx.Err())
}
func TestCommand_Help(t *testing.T) {
t.Parallel()
cmd := func() *clibase.Cmd {
return &clibase.Cmd{
Use: "root",
HelpHandler: (func(i *clibase.Invocation) error {
i.Stdout.Write([]byte("abdracadabra"))
return nil
}),
Handler: (func(i *clibase.Invocation) error {
return xerrors.New("should not be called")
}),
}
}
t.Run("NoHandler", func(t *testing.T) {
t.Parallel()
c := cmd()
c.HelpHandler = nil
err := c.Invoke("--help").Run()
require.Error(t, err)
})
t.Run("Long", func(t *testing.T) {
t.Parallel()
inv := cmd().Invoke("--help")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Contains(t, stdio.Stdout.String(), "abdracadabra")
})
t.Run("Short", func(t *testing.T) {
t.Parallel()
inv := cmd().Invoke("-h")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Contains(t, stdio.Stdout.String(), "abdracadabra")
})
}
func TestCommand_SliceFlags(t *testing.T) {
t.Parallel()
cmd := func(want ...string) *clibase.Cmd {
var got []string
return &clibase.Cmd{
Use: "root",
Options: clibase.OptionSet{
{
Name: "arr",
Flag: "arr",
Default: "bad,bad,bad",
Value: clibase.StringArrayOf(&got),
},
},
Handler: (func(i *clibase.Invocation) error {
require.Equal(t, want, got)
return nil
}),
}
}
err := cmd("good", "good", "good").Invoke("--arr", "good", "--arr", "good", "--arr", "good").Run()
require.NoError(t, err)
err = cmd("bad", "bad", "bad").Invoke().Run()
require.NoError(t, err)
}
func TestCommand_EmptySlice(t *testing.T) {
t.Parallel()
cmd := func(want ...string) *clibase.Cmd {
var got []string
return &clibase.Cmd{
Use: "root",
Options: clibase.OptionSet{
{
Name: "arr",
Flag: "arr",
Default: "def,def,def",
Env: "ARR",
Value: clibase.StringArrayOf(&got),
},
},
Handler: (func(i *clibase.Invocation) error {
require.Equal(t, want, got)
return nil
}),
}
}
// Base-case, uses default.
err := cmd("def", "def", "def").Invoke().Run()
require.NoError(t, err)
// Empty-env uses default, too.
inv := cmd("def", "def", "def").Invoke()
inv.Environ.Set("ARR", "")
require.NoError(t, err)
// Reset to nothing at all via flag.
inv = cmd().Invoke("--arr", "")
inv.Environ.Set("ARR", "cant see")
err = inv.Run()
require.NoError(t, err)
// Reset to a specific value with flag.
inv = cmd("great").Invoke("--arr", "great")
inv.Environ.Set("ARR", "")
err = inv.Run()
require.NoError(t, err)
}
func TestCommand_DefaultsOverride(t *testing.T) {
t.Parallel()
test := func(name string, want string, fn func(t *testing.T, inv *clibase.Invocation)) {
t.Run(name, func(t *testing.T) {
t.Parallel()
var (
got string
config clibase.YAMLConfigPath
)
cmd := &clibase.Cmd{
Options: clibase.OptionSet{
{
Name: "url",
Flag: "url",
Default: "def.com",
Env: "URL",
Value: clibase.StringOf(&got),
YAML: "url",
},
{
Name: "config",
Flag: "config",
Default: "",
Value: &config,
},
},
Handler: (func(i *clibase.Invocation) error {
_, _ = fmt.Fprintf(i.Stdout, "%s", got)
return nil
}),
}
inv := cmd.Invoke()
stdio := fakeIO(inv)
fn(t, inv)
err := inv.Run()
require.NoError(t, err)
require.Equal(t, want, stdio.Stdout.String())
})
}
test("DefaultOverNothing", "def.com", func(t *testing.T, inv *clibase.Invocation) {})
test("FlagOverDefault", "good.com", func(t *testing.T, inv *clibase.Invocation) {
inv.Args = []string{"--url", "good.com"}
})
test("EnvOverDefault", "good.com", func(t *testing.T, inv *clibase.Invocation) {
inv.Environ.Set("URL", "good.com")
})
test("FlagOverEnv", "good.com", func(t *testing.T, inv *clibase.Invocation) {
inv.Environ.Set("URL", "bad.com")
inv.Args = []string{"--url", "good.com"}
})
test("FlagOverYAML", "good.com", func(t *testing.T, inv *clibase.Invocation) {
fi, err := os.CreateTemp(t.TempDir(), "config.yaml")
require.NoError(t, err)
defer fi.Close()
_, err = fi.WriteString("url: bad.com")
require.NoError(t, err)
inv.Args = []string{"--config", fi.Name(), "--url", "good.com"}
})
test("YAMLOverDefault", "good.com", func(t *testing.T, inv *clibase.Invocation) {
fi, err := os.CreateTemp(t.TempDir(), "config.yaml")
require.NoError(t, err)
defer fi.Close()
_, err = fi.WriteString("url: good.com")
require.NoError(t, err)
inv.Args = []string{"--config", fi.Name()}
})
}
-76
View File
@@ -1,76 +0,0 @@
package clibase
import "strings"
// name returns the name of the environment variable.
func envName(line string) string {
return strings.ToUpper(
strings.SplitN(line, "=", 2)[0],
)
}
// value returns the value of the environment variable.
func envValue(line string) string {
tokens := strings.SplitN(line, "=", 2)
if len(tokens) < 2 {
return ""
}
return tokens[1]
}
// Var represents a single environment variable of form
// NAME=VALUE.
type EnvVar struct {
Name string
Value string
}
type Environ []EnvVar
func (e Environ) ToOS() []string {
var env []string
for _, v := range e {
env = append(env, v.Name+"="+v.Value)
}
return env
}
func (e Environ) Lookup(name string) (string, bool) {
for _, v := range e {
if v.Name == name {
return v.Value, true
}
}
return "", false
}
func (e Environ) Get(name string) string {
v, _ := e.Lookup(name)
return v
}
func (e *Environ) Set(name, value string) {
for i, v := range *e {
if v.Name == name {
(*e)[i].Value = value
return
}
}
*e = append(*e, EnvVar{Name: name, Value: value})
}
// ParseEnviron returns all environment variables starting with
// prefix without said prefix.
func ParseEnviron(environ []string, prefix string) Environ {
var filtered []EnvVar
for _, line := range environ {
name := envName(line)
if strings.HasPrefix(name, prefix) {
filtered = append(filtered, EnvVar{
Name: strings.TrimPrefix(name, prefix),
Value: envValue(line),
})
}
}
return filtered
}
-44
View File
@@ -1,44 +0,0 @@
package clibase_test
import (
"reflect"
"testing"
"github.com/coder/coder/v2/cli/clibase"
)
func TestFilterNamePrefix(t *testing.T) {
t.Parallel()
type args struct {
environ []string
prefix string
}
tests := []struct {
name string
args args
want clibase.Environ
}{
{"empty", args{[]string{}, "SHIRE"}, nil},
{
"ONE",
args{
[]string{
"SHIRE_BRANDYBUCK=hmm",
},
"SHIRE_",
},
[]clibase.EnvVar{
{Name: "BRANDYBUCK", Value: "hmm"},
},
},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
if got := clibase.ParseEnviron(tt.args.environ, tt.args.prefix); !reflect.DeepEqual(got, tt.want) {
t.Errorf("FilterNamePrefix() = %v, want %v", got, tt.want)
}
})
}
}
-50
View File
@@ -1,50 +0,0 @@
package clibase
import (
"net"
"strconv"
"github.com/pion/udp"
"golang.org/x/xerrors"
)
// Net abstracts CLI commands interacting with the operating system networking.
//
// At present, it covers opening local listening sockets, since doing this
// in testing is a challenge without flakes, since it's hard to pick a port we
// know a priori will be free.
type Net interface {
// Listen has the same semantics as `net.Listen` but also supports `udp`
Listen(network, address string) (net.Listener, error)
}
// osNet is an implementation that call the real OS for networking.
type osNet struct{}
func (osNet) Listen(network, address string) (net.Listener, error) {
switch network {
case "tcp", "tcp4", "tcp6", "unix", "unixpacket":
return net.Listen(network, address)
case "udp":
host, port, err := net.SplitHostPort(address)
if err != nil {
return nil, xerrors.Errorf("split %q: %w", address, err)
}
var portInt int
portInt, err = strconv.Atoi(port)
if err != nil {
return nil, xerrors.Errorf("parse port %v from %q as int: %w", port, address, err)
}
// Use pion here so that we get a stream-style net.Conn listener, instead
// of a packet-oriented connection that can read and write to multiple
// addresses.
return udp.Listen(network, &net.UDPAddr{
IP: net.ParseIP(host),
Port: portInt,
})
default:
return nil, xerrors.Errorf("unknown listen network %q", network)
}
}
-346
View File
@@ -1,346 +0,0 @@
package clibase
import (
"bytes"
"encoding/json"
"os"
"strings"
"github.com/hashicorp/go-multierror"
"github.com/spf13/pflag"
"golang.org/x/xerrors"
)
type ValueSource string
const (
ValueSourceNone ValueSource = ""
ValueSourceFlag ValueSource = "flag"
ValueSourceEnv ValueSource = "env"
ValueSourceYAML ValueSource = "yaml"
ValueSourceDefault ValueSource = "default"
)
// Option is a configuration option for a CLI application.
type Option struct {
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
// Required means this value must be set by some means. It requires
// `ValueSource != ValueSourceNone`
// If `Default` is set, then `Required` is ignored.
Required bool `json:"required,omitempty"`
// Flag is the long name of the flag used to configure this option. If unset,
// flag configuring is disabled.
Flag string `json:"flag,omitempty"`
// FlagShorthand is the one-character shorthand for the flag. If unset, no
// shorthand is used.
FlagShorthand string `json:"flag_shorthand,omitempty"`
// Env is the environment variable used to configure this option. If unset,
// environment configuring is disabled.
Env string `json:"env,omitempty"`
// YAML is the YAML key used to configure this option. If unset, YAML
// configuring is disabled.
YAML string `json:"yaml,omitempty"`
// Default is parsed into Value if set.
Default string `json:"default,omitempty"`
// Value includes the types listed in values.go.
Value pflag.Value `json:"value,omitempty"`
// Annotations enable extensions to clibase higher up in the stack. It's useful for
// help formatting and documentation generation.
Annotations Annotations `json:"annotations,omitempty"`
// Group is a group hierarchy that helps organize this option in help, configs
// and other documentation.
Group *Group `json:"group,omitempty"`
// UseInstead is a list of options that should be used instead of this one.
// The field is used to generate a deprecation warning.
UseInstead []Option `json:"use_instead,omitempty"`
Hidden bool `json:"hidden,omitempty"`
ValueSource ValueSource `json:"value_source,omitempty"`
}
// optionNoMethods is just a wrapper around Option so we can defer to the
// default json.Unmarshaler behavior.
type optionNoMethods Option
func (o *Option) UnmarshalJSON(data []byte) error {
// If an option has no values, we have no idea how to unmarshal it.
// So just discard the json data.
if o.Value == nil {
o.Value = &DiscardValue
}
return json.Unmarshal(data, (*optionNoMethods)(o))
}
func (o Option) YAMLPath() string {
if o.YAML == "" {
return ""
}
var gs []string
for _, g := range o.Group.Ancestry() {
gs = append(gs, g.YAML)
}
return strings.Join(append(gs, o.YAML), ".")
}
// OptionSet is a group of options that can be applied to a command.
type OptionSet []Option
// UnmarshalJSON implements json.Unmarshaler for OptionSets. Options have an
// interface Value type that cannot handle unmarshalling because the types cannot
// be inferred. Since it is a slice, instantiating the Options first does not
// help.
//
// However, we typically do instantiate the slice to have the correct types.
// So this unmarshaller will attempt to find the named option in the existing
// set, if it cannot, the value is discarded. If the option exists, the value
// is unmarshalled into the existing option, and replaces the existing option.
//
// The value is discarded if it's type cannot be inferred. This behavior just
// feels "safer", although it should never happen if the correct option set
// is passed in. The situation where this could occur is if a client and server
// are on different versions with different options.
func (optSet *OptionSet) UnmarshalJSON(data []byte) error {
dec := json.NewDecoder(bytes.NewBuffer(data))
// Should be a json array, so consume the starting open bracket.
t, err := dec.Token()
if err != nil {
return xerrors.Errorf("read array open bracket: %w", err)
}
if t != json.Delim('[') {
return xerrors.Errorf("expected array open bracket, got %q", t)
}
// As long as json elements exist, consume them. The counter is used for
// better errors.
var i int
OptionSetDecodeLoop:
for dec.More() {
var opt Option
// jValue is a placeholder value that allows us to capture the
// raw json for the value to attempt to unmarshal later.
var jValue jsonValue
opt.Value = &jValue
err := dec.Decode(&opt)
if err != nil {
return xerrors.Errorf("decode %d option: %w", i, err)
}
// This counter is used to contextualize errors to show which element of
// the array we failed to decode. It is only used in the error above, as
// if the above works, we can instead use the Option.Name which is more
// descriptive and useful. So increment here for the next decode.
i++
// Try to see if the option already exists in the option set.
// If it does, just update the existing option.
for optIndex, have := range *optSet {
if have.Name == opt.Name {
if jValue != nil {
err := json.Unmarshal(jValue, &(*optSet)[optIndex].Value)
if err != nil {
return xerrors.Errorf("decode option %q value: %w", have.Name, err)
}
// Set the opt's value
opt.Value = (*optSet)[optIndex].Value
} else {
// Hopefully the user passed empty values in the option set. There is no easy way
// to tell, and if we do not do this, it breaks json.Marshal if we do it again on
// this new option set.
opt.Value = (*optSet)[optIndex].Value
}
// Override the existing.
(*optSet)[optIndex] = opt
// Go to the next option to decode.
continue OptionSetDecodeLoop
}
}
// If the option doesn't exist, the value will be discarded.
// We do this because we cannot infer the type of the value.
opt.Value = DiscardValue
*optSet = append(*optSet, opt)
}
t, err = dec.Token()
if err != nil {
return xerrors.Errorf("read array close bracket: %w", err)
}
if t != json.Delim(']') {
return xerrors.Errorf("expected array close bracket, got %q", t)
}
return nil
}
// Add adds the given Options to the OptionSet.
func (optSet *OptionSet) Add(opts ...Option) {
*optSet = append(*optSet, opts...)
}
// Filter will only return options that match the given filter. (return true)
func (optSet OptionSet) Filter(filter func(opt Option) bool) OptionSet {
cpy := make(OptionSet, 0)
for _, opt := range optSet {
if filter(opt) {
cpy = append(cpy, opt)
}
}
return cpy
}
// FlagSet returns a pflag.FlagSet for the OptionSet.
func (optSet *OptionSet) FlagSet() *pflag.FlagSet {
if optSet == nil {
return &pflag.FlagSet{}
}
fs := pflag.NewFlagSet("", pflag.ContinueOnError)
for _, opt := range *optSet {
if opt.Flag == "" {
continue
}
var noOptDefValue string
{
no, ok := opt.Value.(NoOptDefValuer)
if ok {
noOptDefValue = no.NoOptDefValue()
}
}
val := opt.Value
if val == nil {
val = DiscardValue
}
fs.AddFlag(&pflag.Flag{
Name: opt.Flag,
Shorthand: opt.FlagShorthand,
Usage: opt.Description,
Value: val,
DefValue: "",
Changed: false,
Deprecated: "",
NoOptDefVal: noOptDefValue,
Hidden: opt.Hidden,
})
}
fs.Usage = func() {
_, _ = os.Stderr.WriteString("Override (*FlagSet).Usage() to print help text.\n")
}
return fs
}
// ParseEnv parses the given environment variables into the OptionSet.
// Use EnvsWithPrefix to filter out prefixes.
func (optSet *OptionSet) ParseEnv(vs []EnvVar) error {
if optSet == nil {
return nil
}
var merr *multierror.Error
// We parse environment variables first instead of using a nested loop to
// avoid N*M complexity when there are a lot of options and environment
// variables.
envs := make(map[string]string)
for _, v := range vs {
envs[v.Name] = v.Value
}
for i, opt := range *optSet {
if opt.Env == "" {
continue
}
envVal, ok := envs[opt.Env]
if !ok {
// Homebrew strips all environment variables that do not start with `HOMEBREW_`.
// This prevented using brew to invoke the Coder agent, because the environment
// variables to not get passed down.
//
// A customer wanted to use their custom tap inside a workspace, which was failing
// because the agent lacked the environment variables to authenticate with Git.
envVal, ok = envs[`HOMEBREW_`+opt.Env]
}
// Currently, empty values are treated as if the environment variable is
// unset. This behavior is technically not correct as there is now no
// way for a user to change a Default value to an empty string from
// the environment. Unfortunately, we have old configuration files
// that rely on the faulty behavior.
//
// TODO: We should remove this hack in May 2023, when deployments
// have had months to migrate to the new behavior.
if !ok || envVal == "" {
continue
}
(*optSet)[i].ValueSource = ValueSourceEnv
if err := opt.Value.Set(envVal); err != nil {
merr = multierror.Append(
merr, xerrors.Errorf("parse %q: %w", opt.Name, err),
)
}
}
return merr.ErrorOrNil()
}
// SetDefaults sets the default values for each Option, skipping values
// that already have a value source.
func (optSet *OptionSet) SetDefaults() error {
if optSet == nil {
return nil
}
var merr *multierror.Error
for i, opt := range *optSet {
// Skip values that may have already been set by the user.
if opt.ValueSource != ValueSourceNone {
continue
}
if opt.Default == "" {
continue
}
if opt.Value == nil {
merr = multierror.Append(
merr,
xerrors.Errorf(
"parse %q: no Value field set\nFull opt: %+v",
opt.Name, opt,
),
)
continue
}
(*optSet)[i].ValueSource = ValueSourceDefault
if err := opt.Value.Set(opt.Default); err != nil {
merr = multierror.Append(
merr, xerrors.Errorf("parse %q: %w", opt.Name, err),
)
}
}
return merr.ErrorOrNil()
}
// ByName returns the Option with the given name, or nil if no such option
// exists.
func (optSet *OptionSet) ByName(name string) *Option {
for i := range *optSet {
opt := &(*optSet)[i]
if opt.Name == name {
return opt
}
}
return nil
}
-391
View File
@@ -1,391 +0,0 @@
package clibase_test
import (
"bytes"
"encoding/json"
"regexp"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/codersdk"
)
func TestOptionSet_ParseFlags(t *testing.T) {
t.Parallel()
t.Run("SimpleString", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Flag: "workspace-name",
FlagShorthand: "n",
},
}
var err error
err = os.FlagSet().Parse([]string{"--workspace-name", "foo"})
require.NoError(t, err)
require.EqualValues(t, "foo", workspaceName)
err = os.FlagSet().Parse([]string{"-n", "f"})
require.NoError(t, err)
require.EqualValues(t, "f", workspaceName)
})
t.Run("StringArray", func(t *testing.T) {
t.Parallel()
var names clibase.StringArray
os := clibase.OptionSet{
clibase.Option{
Name: "name",
Value: &names,
Flag: "name",
FlagShorthand: "n",
},
}
err := os.SetDefaults()
require.NoError(t, err)
err = os.FlagSet().Parse([]string{"--name", "foo", "--name", "bar"})
require.NoError(t, err)
require.EqualValues(t, []string{"foo", "bar"}, names)
})
t.Run("ExtraFlags", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
},
}
err := os.FlagSet().Parse([]string{"--some-unknown", "foo"})
require.Error(t, err)
})
t.Run("RegexValid", func(t *testing.T) {
t.Parallel()
var regexpString clibase.Regexp
os := clibase.OptionSet{
clibase.Option{
Name: "RegexpString",
Value: &regexpString,
Flag: "regexp-string",
},
}
err := os.FlagSet().Parse([]string{"--regexp-string", "$test^"})
require.NoError(t, err)
})
t.Run("RegexInvalid", func(t *testing.T) {
t.Parallel()
var regexpString clibase.Regexp
os := clibase.OptionSet{
clibase.Option{
Name: "RegexpString",
Value: &regexpString,
Flag: "regexp-string",
},
}
err := os.FlagSet().Parse([]string{"--regexp-string", "(("})
require.Error(t, err)
})
}
func TestOptionSet_ParseEnv(t *testing.T) {
t.Parallel()
t.Run("SimpleString", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Env: "WORKSPACE_NAME",
},
}
err := os.ParseEnv([]clibase.EnvVar{
{Name: "WORKSPACE_NAME", Value: "foo"},
})
require.NoError(t, err)
require.EqualValues(t, "foo", workspaceName)
})
t.Run("EmptyValue", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Default: "defname",
Env: "WORKSPACE_NAME",
},
}
err := os.SetDefaults()
require.NoError(t, err)
err = os.ParseEnv(clibase.ParseEnviron([]string{"CODER_WORKSPACE_NAME="}, "CODER_"))
require.NoError(t, err)
require.EqualValues(t, "defname", workspaceName)
})
t.Run("StringSlice", func(t *testing.T) {
t.Parallel()
var actual clibase.StringArray
expected := []string{"foo", "bar", "baz"}
os := clibase.OptionSet{
clibase.Option{
Name: "name",
Value: &actual,
Env: "NAMES",
},
}
err := os.SetDefaults()
require.NoError(t, err)
err = os.ParseEnv([]clibase.EnvVar{
{Name: "NAMES", Value: "foo,bar,baz"},
})
require.NoError(t, err)
require.EqualValues(t, expected, actual)
})
t.Run("StructMapStringString", func(t *testing.T) {
t.Parallel()
var actual clibase.Struct[map[string]string]
expected := map[string]string{"foo": "bar", "baz": "zap"}
os := clibase.OptionSet{
clibase.Option{
Name: "labels",
Value: &actual,
Env: "LABELS",
},
}
err := os.SetDefaults()
require.NoError(t, err)
err = os.ParseEnv([]clibase.EnvVar{
{Name: "LABELS", Value: `{"foo":"bar","baz":"zap"}`},
})
require.NoError(t, err)
require.EqualValues(t, expected, actual.Value)
})
t.Run("Homebrew", func(t *testing.T) {
t.Parallel()
var agentToken clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Agent Token",
Value: &agentToken,
Env: "AGENT_TOKEN",
},
}
err := os.ParseEnv([]clibase.EnvVar{
{Name: "HOMEBREW_AGENT_TOKEN", Value: "foo"},
})
require.NoError(t, err)
require.EqualValues(t, "foo", agentToken)
})
}
func TestOptionSet_JsonMarshal(t *testing.T) {
t.Parallel()
// This unit test ensures if the source optionset is missing the option
// and cannot determine the type, it will not panic. The unmarshal will
// succeed with a best effort.
t.Run("MissingSrcOption", func(t *testing.T) {
t.Parallel()
var str clibase.String = "something"
var arr clibase.StringArray = []string{"foo", "bar"}
opts := clibase.OptionSet{
clibase.Option{
Name: "StringOpt",
Value: &str,
},
clibase.Option{
Name: "ArrayOpt",
Value: &arr,
},
}
data, err := json.Marshal(opts)
require.NoError(t, err, "marshal option set")
tgt := clibase.OptionSet{}
err = json.Unmarshal(data, &tgt)
require.NoError(t, err, "unmarshal option set")
for i := range opts {
compareOptionsExceptValues(t, opts[i], tgt[i])
require.Empty(t, tgt[i].Value.String(), "unknown value types are empty")
}
})
t.Run("RegexCase", func(t *testing.T) {
t.Parallel()
val := clibase.Regexp(*regexp.MustCompile(".*"))
opts := clibase.OptionSet{
clibase.Option{
Name: "Regex",
Value: &val,
Default: ".*",
},
}
data, err := json.Marshal(opts)
require.NoError(t, err, "marshal option set")
var foundVal clibase.Regexp
newOpts := clibase.OptionSet{
clibase.Option{
Name: "Regex",
Value: &foundVal,
},
}
err = json.Unmarshal(data, &newOpts)
require.NoError(t, err, "unmarshal option set")
require.EqualValues(t, opts[0].Value.String(), newOpts[0].Value.String())
})
t.Run("AllValues", func(t *testing.T) {
t.Parallel()
vals := coderdtest.DeploymentValues(t)
opts := vals.Options()
sources := []clibase.ValueSource{
clibase.ValueSourceNone,
clibase.ValueSourceFlag,
clibase.ValueSourceEnv,
clibase.ValueSourceYAML,
clibase.ValueSourceDefault,
}
for i := range opts {
opts[i].ValueSource = sources[i%len(sources)]
}
data, err := json.Marshal(opts)
require.NoError(t, err, "marshal option set")
newOpts := (&codersdk.DeploymentValues{}).Options()
err = json.Unmarshal(data, &newOpts)
require.NoError(t, err, "unmarshal option set")
for i := range opts {
exp := opts[i]
found := newOpts[i]
compareOptionsExceptValues(t, exp, found)
compareValues(t, exp, found)
}
thirdOpts := (&codersdk.DeploymentValues{}).Options()
data, err = json.Marshal(newOpts)
require.NoError(t, err, "marshal option set")
err = json.Unmarshal(data, &thirdOpts)
require.NoError(t, err, "unmarshal option set")
// Compare to the original opts again
for i := range opts {
exp := opts[i]
found := thirdOpts[i]
compareOptionsExceptValues(t, exp, found)
compareValues(t, exp, found)
}
})
}
func compareOptionsExceptValues(t *testing.T, exp, found clibase.Option) {
t.Helper()
require.Equalf(t, exp.Name, found.Name, "option name %q", exp.Name)
require.Equalf(t, exp.Description, found.Description, "option description %q", exp.Name)
require.Equalf(t, exp.Required, found.Required, "option required %q", exp.Name)
require.Equalf(t, exp.Flag, found.Flag, "option flag %q", exp.Name)
require.Equalf(t, exp.FlagShorthand, found.FlagShorthand, "option flag shorthand %q", exp.Name)
require.Equalf(t, exp.Env, found.Env, "option env %q", exp.Name)
require.Equalf(t, exp.YAML, found.YAML, "option yaml %q", exp.Name)
require.Equalf(t, exp.Default, found.Default, "option default %q", exp.Name)
require.Equalf(t, exp.ValueSource, found.ValueSource, "option value source %q", exp.Name)
require.Equalf(t, exp.Hidden, found.Hidden, "option hidden %q", exp.Name)
require.Equalf(t, exp.Annotations, found.Annotations, "option annotations %q", exp.Name)
require.Equalf(t, exp.Group, found.Group, "option group %q", exp.Name)
// UseInstead is the same comparison problem, just check the length
require.Equalf(t, len(exp.UseInstead), len(found.UseInstead), "option use instead %q", exp.Name)
}
func compareValues(t *testing.T, exp, found clibase.Option) {
t.Helper()
if (exp.Value == nil || found.Value == nil) || (exp.Value.String() != found.Value.String() && found.Value.String() == "") {
// If the string values are different, this can be a "nil" issue.
// So only run this case if the found string is the empty string.
// We use MarshalYAML for struct strings, and it will return an
// empty string '""' for nil slices/maps/etc.
// So use json to compare.
expJSON, err := json.Marshal(exp.Value)
require.NoError(t, err, "marshal")
foundJSON, err := json.Marshal(found.Value)
require.NoError(t, err, "marshal")
expJSON = normalizeJSON(expJSON)
foundJSON = normalizeJSON(foundJSON)
assert.Equalf(t, string(expJSON), string(foundJSON), "option value %q", exp.Name)
} else {
assert.Equal(t,
exp.Value.String(),
found.Value.String(),
"option value %q", exp.Name)
}
}
// normalizeJSON handles the fact that an empty map/slice is not the same
// as a nil empty/slice. For our purposes, they are the same.
func normalizeJSON(data []byte) []byte {
if bytes.Equal(data, []byte("[]")) || bytes.Equal(data, []byte("{}")) {
return []byte("null")
}
return data
}
-593
View File
@@ -1,593 +0,0 @@
package clibase
import (
"encoding/csv"
"encoding/json"
"fmt"
"net"
"net/url"
"reflect"
"regexp"
"strconv"
"strings"
"time"
"github.com/spf13/pflag"
"golang.org/x/xerrors"
"gopkg.in/yaml.v3"
)
// NoOptDefValuer describes behavior when no
// option is passed into the flag.
//
// This is useful for boolean or otherwise binary flags.
type NoOptDefValuer interface {
NoOptDefValue() string
}
// Validator is a wrapper around a pflag.Value that allows for validation
// of the value after or before it has been set.
type Validator[T pflag.Value] struct {
Value T
// validate is called after the value is set.
validate func(T) error
}
func Validate[T pflag.Value](opt T, validate func(value T) error) *Validator[T] {
return &Validator[T]{Value: opt, validate: validate}
}
func (i *Validator[T]) String() string {
return i.Value.String()
}
func (i *Validator[T]) Set(input string) error {
err := i.Value.Set(input)
if err != nil {
return err
}
if i.validate != nil {
err = i.validate(i.Value)
if err != nil {
return err
}
}
return nil
}
func (i *Validator[T]) Type() string {
return i.Value.Type()
}
func (i *Validator[T]) MarshalYAML() (interface{}, error) {
m, ok := any(i.Value).(yaml.Marshaler)
if !ok {
return i.Value, nil
}
return m.MarshalYAML()
}
func (i *Validator[T]) UnmarshalYAML(n *yaml.Node) error {
return n.Decode(i.Value)
}
func (i *Validator[T]) MarshalJSON() ([]byte, error) {
return json.Marshal(i.Value)
}
func (i *Validator[T]) UnmarshalJSON(b []byte) error {
return json.Unmarshal(b, i.Value)
}
func (i *Validator[T]) Underlying() pflag.Value { return i.Value }
// values.go contains a standard set of value types that can be used as
// Option Values.
type Int64 int64
func Int64Of(i *int64) *Int64 {
return (*Int64)(i)
}
func (i *Int64) Set(s string) error {
ii, err := strconv.ParseInt(s, 10, 64)
*i = Int64(ii)
return err
}
func (i Int64) Value() int64 {
return int64(i)
}
func (i Int64) String() string {
return strconv.Itoa(int(i))
}
func (Int64) Type() string {
return "int"
}
type Bool bool
func BoolOf(b *bool) *Bool {
return (*Bool)(b)
}
func (b *Bool) Set(s string) error {
if s == "" {
*b = Bool(false)
return nil
}
bb, err := strconv.ParseBool(s)
*b = Bool(bb)
return err
}
func (*Bool) NoOptDefValue() string {
return "true"
}
func (b Bool) String() string {
return strconv.FormatBool(bool(b))
}
func (b Bool) Value() bool {
return bool(b)
}
func (Bool) Type() string {
return "bool"
}
type String string
func StringOf(s *string) *String {
return (*String)(s)
}
func (*String) NoOptDefValue() string {
return ""
}
func (s *String) Set(v string) error {
*s = String(v)
return nil
}
func (s String) String() string {
return string(s)
}
func (s String) Value() string {
return string(s)
}
func (String) Type() string {
return "string"
}
var _ pflag.SliceValue = &StringArray{}
// StringArray is a slice of strings that implements pflag.Value and pflag.SliceValue.
type StringArray []string
func StringArrayOf(ss *[]string) *StringArray {
return (*StringArray)(ss)
}
func (s *StringArray) Append(v string) error {
*s = append(*s, v)
return nil
}
func (s *StringArray) Replace(vals []string) error {
*s = vals
return nil
}
func (s *StringArray) GetSlice() []string {
return *s
}
func readAsCSV(v string) ([]string, error) {
return csv.NewReader(strings.NewReader(v)).Read()
}
func writeAsCSV(vals []string) string {
var sb strings.Builder
err := csv.NewWriter(&sb).Write(vals)
if err != nil {
return fmt.Sprintf("error: %s", err)
}
return sb.String()
}
func (s *StringArray) Set(v string) error {
if v == "" {
*s = nil
return nil
}
ss, err := readAsCSV(v)
if err != nil {
return err
}
*s = append(*s, ss...)
return nil
}
func (s StringArray) String() string {
return writeAsCSV([]string(s))
}
func (s StringArray) Value() []string {
return []string(s)
}
func (StringArray) Type() string {
return "string-array"
}
type Duration time.Duration
func DurationOf(d *time.Duration) *Duration {
return (*Duration)(d)
}
func (d *Duration) Set(v string) error {
dd, err := time.ParseDuration(v)
*d = Duration(dd)
return err
}
func (d *Duration) Value() time.Duration {
return time.Duration(*d)
}
func (d *Duration) String() string {
return time.Duration(*d).String()
}
func (Duration) Type() string {
return "duration"
}
func (d *Duration) MarshalYAML() (interface{}, error) {
return yaml.Node{
Kind: yaml.ScalarNode,
Value: d.String(),
}, nil
}
func (d *Duration) UnmarshalYAML(n *yaml.Node) error {
return d.Set(n.Value)
}
type URL url.URL
func URLOf(u *url.URL) *URL {
return (*URL)(u)
}
func (u *URL) Set(v string) error {
uu, err := url.Parse(v)
if err != nil {
return err
}
*u = URL(*uu)
return nil
}
func (u *URL) String() string {
uu := url.URL(*u)
return uu.String()
}
func (u *URL) MarshalYAML() (interface{}, error) {
return yaml.Node{
Kind: yaml.ScalarNode,
Value: u.String(),
}, nil
}
func (u *URL) UnmarshalYAML(n *yaml.Node) error {
return u.Set(n.Value)
}
func (u *URL) MarshalJSON() ([]byte, error) {
return json.Marshal(u.String())
}
func (u *URL) UnmarshalJSON(b []byte) error {
var s string
err := json.Unmarshal(b, &s)
if err != nil {
return err
}
return u.Set(s)
}
func (*URL) Type() string {
return "url"
}
func (u *URL) Value() *url.URL {
return (*url.URL)(u)
}
// HostPort is a host:port pair.
type HostPort struct {
Host string
Port string
}
func (hp *HostPort) Set(v string) error {
if v == "" {
return xerrors.Errorf("must not be empty")
}
var err error
hp.Host, hp.Port, err = net.SplitHostPort(v)
return err
}
func (hp *HostPort) String() string {
if hp.Host == "" && hp.Port == "" {
return ""
}
// Warning: net.JoinHostPort must be used over concatenation to support
// IPv6 addresses.
return net.JoinHostPort(hp.Host, hp.Port)
}
func (hp *HostPort) MarshalJSON() ([]byte, error) {
return json.Marshal(hp.String())
}
func (hp *HostPort) UnmarshalJSON(b []byte) error {
var s string
err := json.Unmarshal(b, &s)
if err != nil {
return err
}
if s == "" {
hp.Host = ""
hp.Port = ""
return nil
}
return hp.Set(s)
}
func (hp *HostPort) MarshalYAML() (interface{}, error) {
return yaml.Node{
Kind: yaml.ScalarNode,
Value: hp.String(),
}, nil
}
func (hp *HostPort) UnmarshalYAML(n *yaml.Node) error {
return hp.Set(n.Value)
}
func (*HostPort) Type() string {
return "host:port"
}
var (
_ yaml.Marshaler = new(Struct[struct{}])
_ yaml.Unmarshaler = new(Struct[struct{}])
)
// Struct is a special value type that encodes an arbitrary struct.
// It implements the flag.Value interface, but in general these values should
// only be accepted via config for ergonomics.
//
// The string encoding type is YAML.
type Struct[T any] struct {
Value T
}
//nolint:revive
func (s *Struct[T]) Set(v string) error {
return yaml.Unmarshal([]byte(v), &s.Value)
}
//nolint:revive
func (s *Struct[T]) String() string {
byt, err := yaml.Marshal(s.Value)
if err != nil {
return "decode failed: " + err.Error()
}
return string(byt)
}
// nolint:revive
func (s *Struct[T]) MarshalYAML() (interface{}, error) {
var n yaml.Node
err := n.Encode(s.Value)
if err != nil {
return nil, err
}
return n, nil
}
// nolint:revive
func (s *Struct[T]) UnmarshalYAML(n *yaml.Node) error {
// HACK: for compatibility with flags, we use nil slices instead of empty
// slices. In most cases, nil slices and empty slices are treated
// the same, so this behavior may be removed at some point.
if typ := reflect.TypeOf(s.Value); typ.Kind() == reflect.Slice && len(n.Content) == 0 {
reflect.ValueOf(&s.Value).Elem().Set(reflect.Zero(typ))
return nil
}
return n.Decode(&s.Value)
}
//nolint:revive
func (s *Struct[T]) Type() string {
return fmt.Sprintf("struct[%T]", s.Value)
}
// nolint:revive
func (s *Struct[T]) MarshalJSON() ([]byte, error) {
return json.Marshal(s.Value)
}
// nolint:revive
func (s *Struct[T]) UnmarshalJSON(b []byte) error {
return json.Unmarshal(b, &s.Value)
}
// DiscardValue does nothing but implements the pflag.Value interface.
// It's useful in cases where you want to accept an option, but access the
// underlying value directly instead of through the Option methods.
var DiscardValue discardValue
type discardValue struct{}
func (discardValue) Set(string) error {
return nil
}
func (discardValue) String() string {
return ""
}
func (discardValue) Type() string {
return "discard"
}
func (discardValue) UnmarshalJSON([]byte) error {
return nil
}
// jsonValue is intentionally not exported. It is just used to store the raw JSON
// data for a value to defer it's unmarshal. It implements the pflag.Value to be
// usable in an Option.
type jsonValue json.RawMessage
func (jsonValue) Set(string) error {
return xerrors.Errorf("json value is read-only")
}
func (jsonValue) String() string {
return ""
}
func (jsonValue) Type() string {
return "json"
}
func (j *jsonValue) UnmarshalJSON(data []byte) error {
if j == nil {
return xerrors.New("json.RawMessage: UnmarshalJSON on nil pointer")
}
*j = append((*j)[0:0], data...)
return nil
}
var _ pflag.Value = (*Enum)(nil)
type Enum struct {
Choices []string
Value *string
}
func EnumOf(v *string, choices ...string) *Enum {
return &Enum{
Choices: choices,
Value: v,
}
}
func (e *Enum) Set(v string) error {
for _, c := range e.Choices {
if v == c {
*e.Value = v
return nil
}
}
return xerrors.Errorf("invalid choice: %s, should be one of %v", v, e.Choices)
}
func (e *Enum) Type() string {
return fmt.Sprintf("enum[%v]", strings.Join(e.Choices, "\\|"))
}
func (e *Enum) String() string {
return *e.Value
}
type Regexp regexp.Regexp
func (r *Regexp) MarshalJSON() ([]byte, error) {
return json.Marshal(r.String())
}
func (r *Regexp) UnmarshalJSON(data []byte) error {
var source string
err := json.Unmarshal(data, &source)
if err != nil {
return err
}
exp, err := regexp.Compile(source)
if err != nil {
return xerrors.Errorf("invalid regex expression: %w", err)
}
*r = Regexp(*exp)
return nil
}
func (r *Regexp) MarshalYAML() (interface{}, error) {
return yaml.Node{
Kind: yaml.ScalarNode,
Value: r.String(),
}, nil
}
func (r *Regexp) UnmarshalYAML(n *yaml.Node) error {
return r.Set(n.Value)
}
func (r *Regexp) Set(v string) error {
exp, err := regexp.Compile(v)
if err != nil {
return xerrors.Errorf("invalid regex expression: %w", err)
}
*r = Regexp(*exp)
return nil
}
func (r Regexp) String() string {
return r.Value().String()
}
func (r *Regexp) Value() *regexp.Regexp {
if r == nil {
return nil
}
return (*regexp.Regexp)(r)
}
func (Regexp) Type() string {
return "regexp"
}
var _ pflag.Value = (*YAMLConfigPath)(nil)
// YAMLConfigPath is a special value type that encodes a path to a YAML
// configuration file where options are read from.
type YAMLConfigPath string
func (p *YAMLConfigPath) Set(v string) error {
*p = YAMLConfigPath(v)
return nil
}
func (p *YAMLConfigPath) String() string {
return string(*p)
}
func (*YAMLConfigPath) Type() string {
return "yaml-config-path"
}
-299
View File
@@ -1,299 +0,0 @@
package clibase
import (
"errors"
"fmt"
"strings"
"github.com/mitchellh/go-wordwrap"
"github.com/spf13/pflag"
"golang.org/x/xerrors"
"gopkg.in/yaml.v3"
)
var (
_ yaml.Marshaler = new(OptionSet)
_ yaml.Unmarshaler = new(OptionSet)
)
// deepMapNode returns the mapping node at the given path,
// creating it if it doesn't exist.
func deepMapNode(n *yaml.Node, path []string, headComment string) *yaml.Node {
if len(path) == 0 {
return n
}
// Name is every two nodes.
for i := 0; i < len(n.Content)-1; i += 2 {
if n.Content[i].Value == path[0] {
// Found matching name, recurse.
return deepMapNode(n.Content[i+1], path[1:], headComment)
}
}
// Not found, create it.
nameNode := yaml.Node{
Kind: yaml.ScalarNode,
Value: path[0],
HeadComment: headComment,
}
valueNode := yaml.Node{
Kind: yaml.MappingNode,
}
n.Content = append(n.Content, &nameNode)
n.Content = append(n.Content, &valueNode)
return deepMapNode(&valueNode, path[1:], headComment)
}
// MarshalYAML converts the option set to a YAML node, that can be
// converted into bytes via yaml.Marshal.
//
// The node is returned to enable post-processing higher up in
// the stack.
//
// It is isomorphic with FromYAML.
func (optSet *OptionSet) MarshalYAML() (any, error) {
root := yaml.Node{
Kind: yaml.MappingNode,
}
for _, opt := range *optSet {
if opt.YAML == "" {
continue
}
defValue := opt.Default
if defValue == "" {
defValue = "<unset>"
}
comment := wordwrap.WrapString(
fmt.Sprintf("%s\n(default: %s, type: %s)", opt.Description, defValue, opt.Value.Type()),
80,
)
nameNode := yaml.Node{
Kind: yaml.ScalarNode,
Value: opt.YAML,
HeadComment: comment,
}
_, isValidator := opt.Value.(interface{ Underlying() pflag.Value })
var valueNode yaml.Node
if opt.Value == nil {
valueNode = yaml.Node{
Kind: yaml.ScalarNode,
Value: "null",
}
} else if m, ok := opt.Value.(yaml.Marshaler); ok && !isValidator {
// Validators do a wrap, and should be handled by the else statement.
v, err := m.MarshalYAML()
if err != nil {
return nil, xerrors.Errorf(
"marshal %q: %w", opt.Name, err,
)
}
valueNode, ok = v.(yaml.Node)
if !ok {
return nil, xerrors.Errorf(
"marshal %q: unexpected underlying type %T",
opt.Name, v,
)
}
} else {
// The all-other types case.
//
// A bit of a hack, we marshal and then unmarshal to get
// the underlying node.
byt, err := yaml.Marshal(opt.Value)
if err != nil {
return nil, xerrors.Errorf(
"marshal %q: %w", opt.Name, err,
)
}
var docNode yaml.Node
err = yaml.Unmarshal(byt, &docNode)
if err != nil {
return nil, xerrors.Errorf(
"unmarshal %q: %w", opt.Name, err,
)
}
if len(docNode.Content) != 1 {
return nil, xerrors.Errorf(
"unmarshal %q: expected one node, got %d",
opt.Name, len(docNode.Content),
)
}
valueNode = *docNode.Content[0]
}
var group []string
for _, g := range opt.Group.Ancestry() {
if g.YAML == "" {
return nil, xerrors.Errorf(
"group yaml name is empty for %q, groups: %+v",
opt.Name,
opt.Group,
)
}
group = append(group, g.YAML)
}
var groupDesc string
if opt.Group != nil {
groupDesc = wordwrap.WrapString(opt.Group.Description, 80)
}
parentValueNode := deepMapNode(
&root, group,
groupDesc,
)
parentValueNode.Content = append(
parentValueNode.Content,
&nameNode,
&valueNode,
)
}
return &root, nil
}
// mapYAMLNodes converts parent into a map with keys of form "group.subgroup.option"
// and values as the corresponding YAML nodes.
func mapYAMLNodes(parent *yaml.Node) (map[string]*yaml.Node, error) {
if parent.Kind != yaml.MappingNode {
return nil, xerrors.Errorf("expected mapping node, got type %v", parent.Kind)
}
if len(parent.Content)%2 != 0 {
return nil, xerrors.Errorf("expected an even number of k/v pairs, got %d", len(parent.Content))
}
var (
key string
m = make(map[string]*yaml.Node, len(parent.Content)/2)
merr error
)
for i, child := range parent.Content {
if i%2 == 0 {
if child.Kind != yaml.ScalarNode {
// We immediately because the rest of the code is bound to fail
// if we don't know to expect a key or a value.
return nil, xerrors.Errorf("expected scalar node for key, got type %v", child.Kind)
}
key = child.Value
continue
}
// We don't know if this is a grouped simple option or complex option,
// so we store both "key" and "group.key". Since we're storing pointers,
// the additional memory is of little concern.
m[key] = child
if child.Kind != yaml.MappingNode {
continue
}
sub, err := mapYAMLNodes(child)
if err != nil {
merr = errors.Join(merr, xerrors.Errorf("mapping node %q: %w", key, err))
continue
}
for k, v := range sub {
m[key+"."+k] = v
}
}
return m, nil
}
func (o *Option) setFromYAMLNode(n *yaml.Node) error {
o.ValueSource = ValueSourceYAML
if um, ok := o.Value.(yaml.Unmarshaler); ok {
return um.UnmarshalYAML(n)
}
switch n.Kind {
case yaml.ScalarNode:
return o.Value.Set(n.Value)
case yaml.SequenceNode:
// We treat empty values as nil for consistency with other option
// mechanisms.
if len(n.Content) == 0 {
o.Value = nil
return nil
}
return n.Decode(o.Value)
case yaml.MappingNode:
return xerrors.Errorf("mapping nodes must implement yaml.Unmarshaler")
default:
return xerrors.Errorf("unexpected node kind %v", n.Kind)
}
}
// UnmarshalYAML converts the given YAML node into the option set.
// It is isomorphic with ToYAML.
func (optSet *OptionSet) UnmarshalYAML(rootNode *yaml.Node) error {
// The rootNode will be a DocumentNode if it's read from a file. We do
// not support multiple documents in a single file.
if rootNode.Kind == yaml.DocumentNode {
if len(rootNode.Content) != 1 {
return xerrors.Errorf("expected one node in document, got %d", len(rootNode.Content))
}
rootNode = rootNode.Content[0]
}
yamlNodes, err := mapYAMLNodes(rootNode)
if err != nil {
return xerrors.Errorf("mapping nodes: %w", err)
}
matchedNodes := make(map[string]*yaml.Node, len(yamlNodes))
var merr error
for i := range *optSet {
opt := &(*optSet)[i]
if opt.YAML == "" {
continue
}
var group []string
for _, g := range opt.Group.Ancestry() {
if g.YAML == "" {
return xerrors.Errorf(
"group yaml name is empty for %q, groups: %+v",
opt.Name,
opt.Group,
)
}
group = append(group, g.YAML)
delete(yamlNodes, strings.Join(group, "."))
}
key := strings.Join(append(group, opt.YAML), ".")
node, ok := yamlNodes[key]
if !ok {
continue
}
matchedNodes[key] = node
if opt.ValueSource != ValueSourceNone {
continue
}
if err := opt.setFromYAMLNode(node); err != nil {
merr = errors.Join(merr, xerrors.Errorf("setting %q: %w", opt.YAML, err))
}
}
// Remove all matched nodes and their descendants from yamlNodes so we
// can accurately report unknown options.
for k := range yamlNodes {
var key string
for _, part := range strings.Split(k, ".") {
if key != "" {
key += "."
}
key += part
if _, ok := matchedNodes[key]; ok {
delete(yamlNodes, k)
}
}
}
for k := range yamlNodes {
merr = errors.Join(merr, xerrors.Errorf("unknown option %q", k))
}
return merr
}
-202
View File
@@ -1,202 +0,0 @@
package clibase_test
import (
"testing"
"github.com/spf13/pflag"
"github.com/stretchr/testify/require"
"golang.org/x/exp/slices"
"gopkg.in/yaml.v3"
"github.com/coder/coder/v2/cli/clibase"
)
func TestOptionSet_YAML(t *testing.T) {
t.Parallel()
t.Run("RequireKey", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Default: "billie",
},
}
node, err := os.MarshalYAML()
require.NoError(t, err)
require.Len(t, node.(*yaml.Node).Content, 0)
})
t.Run("SimpleString", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Default: "billie",
Description: "The workspace's name.",
Group: &clibase.Group{YAML: "names"},
YAML: "workspaceName",
},
}
err := os.SetDefaults()
require.NoError(t, err)
n, err := os.MarshalYAML()
require.NoError(t, err)
// Visually inspect for now.
byt, err := yaml.Marshal(n)
require.NoError(t, err)
t.Logf("Raw YAML:\n%s", string(byt))
})
}
func TestOptionSet_YAMLUnknownOptions(t *testing.T) {
t.Parallel()
os := clibase.OptionSet{
{
Name: "Workspace Name",
Default: "billie",
Description: "The workspace's name.",
YAML: "workspaceName",
Value: new(clibase.String),
},
}
const yamlDoc = `something: else`
err := yaml.Unmarshal([]byte(yamlDoc), &os)
require.Error(t, err)
require.Empty(t, os[0].Value.String())
os[0].YAML = "something"
err = yaml.Unmarshal([]byte(yamlDoc), &os)
require.NoError(t, err)
require.Equal(t, "else", os[0].Value.String())
}
// TestOptionSet_YAMLIsomorphism tests that the YAML representations of an
// OptionSet converts to the same OptionSet when read back in.
func TestOptionSet_YAMLIsomorphism(t *testing.T) {
t.Parallel()
// This is used to form a generic.
//nolint:unused
type kid struct {
Name string `yaml:"name"`
Age int `yaml:"age"`
}
for _, tc := range []struct {
name string
os clibase.OptionSet
zeroValue func() pflag.Value
}{
{
name: "SimpleString",
os: clibase.OptionSet{
{
Name: "Workspace Name",
Default: "billie",
Description: "The workspace's name.",
Group: &clibase.Group{YAML: "names"},
YAML: "workspaceName",
},
},
zeroValue: func() pflag.Value {
return clibase.StringOf(new(string))
},
},
{
name: "Array",
os: clibase.OptionSet{
{
YAML: "names",
Default: "jill,jack,joan",
},
},
zeroValue: func() pflag.Value {
return clibase.StringArrayOf(&[]string{})
},
},
{
name: "ComplexObject",
os: clibase.OptionSet{
{
YAML: "kids",
Default: `- name: jill
age: 12
- name: jack
age: 13`,
},
},
zeroValue: func() pflag.Value {
return &clibase.Struct[[]kid]{}
},
},
{
name: "DeepGroup",
os: clibase.OptionSet{
{
YAML: "names",
Default: "jill,jack,joan",
Group: &clibase.Group{YAML: "kids", Parent: &clibase.Group{YAML: "family"}},
},
},
zeroValue: func() pflag.Value {
return clibase.StringArrayOf(&[]string{})
},
},
} {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
// Set initial values.
for i := range tc.os {
tc.os[i].Value = tc.zeroValue()
}
err := tc.os.SetDefaults()
require.NoError(t, err)
y, err := tc.os.MarshalYAML()
require.NoError(t, err)
toByt, err := yaml.Marshal(y)
require.NoError(t, err)
t.Logf("Raw YAML:\n%s", string(toByt))
var y2 yaml.Node
err = yaml.Unmarshal(toByt, &y2)
require.NoError(t, err)
os2 := slices.Clone(tc.os)
for i := range os2 {
os2[i].Value = tc.zeroValue()
os2[i].ValueSource = clibase.ValueSourceNone
}
// os2 values should be zeroed whereas tc.os should be
// set to defaults.
// This check makes sure we aren't mixing pointers.
require.NotEqual(t, tc.os, os2)
err = os2.UnmarshalYAML(&y2)
require.NoError(t, err)
want := tc.os
for i := range want {
want[i].ValueSource = clibase.ValueSourceYAML
}
require.Equal(t, tc.os, os2)
})
}
}
+2 -2
View File
@@ -14,9 +14,9 @@ import (
"cdr.dev/slog/sloggers/sloghuman"
"cdr.dev/slog/sloggers/slogjson"
"cdr.dev/slog/sloggers/slogstackdriver"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/coderd/tracing"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
type (
@@ -86,7 +86,7 @@ func FromDeploymentValues(vals *codersdk.DeploymentValues) Option {
}
}
func (b *Builder) Build(inv *clibase.Invocation) (log slog.Logger, closeLog func(), err error) {
func (b *Builder) Build(inv *serpent.Invocation) (log slog.Logger, closeLog func(), err error) {
var (
sinks = []slog.Sink{}
closers = []func() error{}
+14 -14
View File
@@ -8,10 +8,10 @@ import (
"strings"
"testing"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/clilog"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -23,7 +23,7 @@ func TestBuilder(t *testing.T) {
t.Run("NoConfiguration", func(t *testing.T) {
t.Parallel()
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t),
}
@@ -35,7 +35,7 @@ func TestBuilder(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "test.log")
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t,
clilog.WithHuman(tempFile),
@@ -51,7 +51,7 @@ func TestBuilder(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "test.log")
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t,
clilog.WithHuman(tempFile),
@@ -68,7 +68,7 @@ func TestBuilder(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "test.log")
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t, clilog.WithHuman(tempFile)),
}
@@ -81,7 +81,7 @@ func TestBuilder(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "test.log")
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t, clilog.WithJSON(tempFile), clilog.WithVerbose()),
}
@@ -107,7 +107,7 @@ func TestBuilder(t *testing.T) {
// Use the default deployment values.
dv := coderdtest.DeploymentValues(t)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t, clilog.FromDeploymentValues(dv)),
}
@@ -127,15 +127,15 @@ func TestBuilder(t *testing.T) {
dv := &codersdk.DeploymentValues{
Logging: codersdk.LoggingConfig{
Filter: []string{"foo", "baz"},
Human: clibase.String(tempFile),
JSON: clibase.String(tempJSON),
Human: serpent.String(tempFile),
JSON: serpent.String(tempJSON),
},
Verbose: true,
Trace: codersdk.TraceConfig{
Enable: true,
},
}
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t, clilog.FromDeploymentValues(dv)),
}
@@ -150,9 +150,9 @@ func TestBuilder(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "doesnotexist", "test.log")
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "test",
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
logger, closeLog, err := clilog.New(
clilog.WithFilter("foo", "baz"),
clilog.WithHuman(tempFile),
@@ -181,10 +181,10 @@ var (
filterLog = "this is an important debug message you want to see"
)
func testHandler(t testing.TB, opts ...clilog.Option) clibase.HandlerFunc {
func testHandler(t testing.TB, opts ...clilog.Option) serpent.HandlerFunc {
t.Helper()
return func(inv *clibase.Invocation) error {
return func(inv *serpent.Invocation) error {
logger, closeLog, err := clilog.New(opts...).Build(inv)
if err != nil {
return err
+18 -8
View File
@@ -20,16 +20,16 @@ import (
"cdr.dev/slog"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/cli"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/config"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/provisioner/echo"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
// New creates a CLI instance with a configuration pointed to a
// temporary testing directory.
func New(t testing.TB, args ...string) (*clibase.Invocation, config.Root) {
func New(t testing.TB, args ...string) (*serpent.Invocation, config.Root) {
var root cli.RootCmd
cmd, err := root.Command(root.AGPL())
@@ -56,15 +56,15 @@ func (l *logWriter) Write(p []byte) (n int, err error) {
}
func NewWithCommand(
t testing.TB, cmd *clibase.Cmd, args ...string,
) (*clibase.Invocation, config.Root) {
t testing.TB, cmd *serpent.Command, args ...string,
) (*serpent.Invocation, config.Root) {
configDir := config.Root(t.TempDir())
// I really would like to fail test on error logs, but realistically, turning on by default
// in all our CLI tests is going to create a lot of flaky noise.
logger := slogtest.Make(t, &slogtest.Options{IgnoreErrors: true}).
Leveled(slog.LevelDebug).
Named("cli")
i := &clibase.Invocation{
i := &serpent.Invocation{
Command: cmd,
Args: append([]string{"--global-config", string(configDir)}, args...),
Stdin: io.LimitReader(nil, 0),
@@ -140,7 +140,11 @@ func extractTar(t *testing.T, data []byte, directory string) {
// Start runs the command in a goroutine and cleans it up when the test
// completed.
func Start(t *testing.T, inv *clibase.Invocation) {
func Start(t *testing.T, inv *serpent.Invocation) {
StartWithAssert(t, inv, nil)
}
func StartWithAssert(t *testing.T, inv *serpent.Invocation, assertCallback func(t *testing.T, err error)) { //nolint:revive
t.Helper()
closeCh := make(chan struct{})
@@ -155,6 +159,12 @@ func Start(t *testing.T, inv *clibase.Invocation) {
go func() {
defer close(closeCh)
err := waiter.Wait()
if assertCallback != nil {
assertCallback(t, err)
return
}
switch {
case errors.Is(err, context.Canceled):
return
@@ -165,7 +175,7 @@ func Start(t *testing.T, inv *clibase.Invocation) {
}
// Run runs the command and asserts that there is no error.
func Run(t *testing.T, inv *clibase.Invocation) {
func Run(t *testing.T, inv *serpent.Invocation) {
t.Helper()
err := inv.Run()
@@ -218,7 +228,7 @@ func (w *ErrorWaiter) RequireAs(want interface{}) {
// StartWithWaiter runs the command in a goroutine but returns the error instead
// of asserting it. This is useful for testing error cases.
func StartWithWaiter(t *testing.T, inv *clibase.Invocation) *ErrorWaiter {
func StartWithWaiter(t *testing.T, inv *serpent.Invocation) *ErrorWaiter {
t.Helper()
var (
+41 -32
View File
@@ -13,12 +13,12 @@ import (
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/config"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/coderd/database/dbtestutil"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
// UpdateGoldenFiles indicates golden files should be updated.
@@ -48,7 +48,7 @@ func DefaultCases() []CommandHelpCase {
// TestCommandHelp will test the help output of the given commands
// using golden files.
func TestCommandHelp(t *testing.T, getRoot func(t *testing.T) *clibase.Cmd, cases []CommandHelpCase) {
func TestCommandHelp(t *testing.T, getRoot func(t *testing.T) *serpent.Command, cases []CommandHelpCase) {
t.Parallel()
rootClient, replacements := prepareTestData(t)
@@ -87,40 +87,45 @@ ExtractCommandPathsLoop:
StartWithWaiter(t, inv.WithContext(ctx)).RequireSuccess()
actual := outBuf.Bytes()
if len(actual) == 0 {
t.Fatal("no output")
}
for k, v := range replacements {
actual = bytes.ReplaceAll(actual, []byte(k), []byte(v))
}
actual = NormalizeGoldenFile(t, actual)
goldenPath := filepath.Join("testdata", strings.Replace(tt.Name, " ", "_", -1)+".golden")
if *UpdateGoldenFiles {
t.Logf("update golden file for: %q: %s", tt.Name, goldenPath)
err := os.WriteFile(goldenPath, actual, 0o600)
require.NoError(t, err, "update golden file")
}
expected, err := os.ReadFile(goldenPath)
require.NoError(t, err, "read golden file, run \"make update-golden-files\" and commit the changes")
expected = NormalizeGoldenFile(t, expected)
require.Equal(
t, string(expected), string(actual),
"golden file mismatch: %s, run \"make update-golden-files\", verify and commit the changes",
goldenPath,
)
TestGoldenFile(t, tt.Name, outBuf.Bytes(), replacements)
})
}
}
// NormalizeGoldenFile replaces any strings that are system or timing dependent
// TestGoldenFile will test the given bytes slice input against the
// golden file with the given file name, optionally using the given replacements.
func TestGoldenFile(t *testing.T, fileName string, actual []byte, replacements map[string]string) {
if len(actual) == 0 {
t.Fatal("no output")
}
for k, v := range replacements {
actual = bytes.ReplaceAll(actual, []byte(k), []byte(v))
}
actual = normalizeGoldenFile(t, actual)
goldenPath := filepath.Join("testdata", strings.ReplaceAll(fileName, " ", "_")+".golden")
if *UpdateGoldenFiles {
t.Logf("update golden file for: %q: %s", fileName, goldenPath)
err := os.WriteFile(goldenPath, actual, 0o600)
require.NoError(t, err, "update golden file")
}
expected, err := os.ReadFile(goldenPath)
require.NoError(t, err, "read golden file, run \"make update-golden-files\" and commit the changes")
expected = normalizeGoldenFile(t, expected)
require.Equal(
t, string(expected), string(actual),
"golden file mismatch: %s, run \"make update-golden-files\", verify and commit the changes",
goldenPath,
)
}
// normalizeGoldenFile replaces any strings that are system or timing dependent
// with a placeholder so that the golden files can be compared with a simple
// equality check.
func NormalizeGoldenFile(t *testing.T, byt []byte) []byte {
func normalizeGoldenFile(t *testing.T, byt []byte) []byte {
// Replace any timestamps with a placeholder.
byt = timestampRegex.ReplaceAll(byt, []byte("[timestamp]"))
@@ -148,7 +153,7 @@ func NormalizeGoldenFile(t *testing.T, byt []byte) []byte {
return byt
}
func extractVisibleCommandPaths(cmdPath []string, cmds []*clibase.Cmd) [][]string {
func extractVisibleCommandPaths(cmdPath []string, cmds []*serpent.Command) [][]string {
var cmdPaths [][]string
for _, c := range cmds {
if c.Hidden {
@@ -167,7 +172,11 @@ func prepareTestData(t *testing.T) (*codersdk.Client, map[string]string) {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitLong)
defer cancel()
db, pubsub := dbtestutil.NewDB(t)
// This needs to be a fixed timezone because timezones increase the length
// of timestamp strings. The increased length can pad table formatting's
// and differ the table header spacings.
//nolint:gocritic
db, pubsub := dbtestutil.NewDB(t, dbtestutil.WithTimezone("UTC"))
rootClient := coderdtest.New(t, &coderdtest.Options{
Database: db,
Pubsub: pubsub,
+4 -4
View File
@@ -3,7 +3,7 @@ package clitest
import (
"testing"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/serpent"
)
// HandlersOK asserts that all commands have a handler.
@@ -11,11 +11,11 @@ import (
// non-root commands (like 'groups' or 'users'), a handler is required.
// These handlers are likely just the 'help' handler, but this must be
// explicitly set.
func HandlersOK(t *testing.T, cmd *clibase.Cmd) {
cmd.Walk(func(cmd *clibase.Cmd) {
func HandlersOK(t *testing.T, cmd *serpent.Command) {
cmd.Walk(func(cmd *serpent.Command) {
if cmd.Handler == nil {
// If you see this error, make the Handler a helper invoker.
// Handler: func(inv *clibase.Invocation) error {
// Handler: func(inv *serpent.Invocation) error {
// return inv.Command.HelpHandler(inv)
// },
t.Errorf("command %q has no handler, change to a helper invoker using: 'inv.Command.HelpHandler(inv)'", cmd.Name())
+59 -3
View File
@@ -2,13 +2,17 @@ package cliui
import (
"context"
"fmt"
"io"
"strconv"
"strings"
"time"
"github.com/google/uuid"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/tailnet"
)
var errAgentShuttingDown = xerrors.New("agent is shutting down")
@@ -208,13 +212,13 @@ func Agent(ctx context.Context, writer io.Writer, agentID uuid.UUID, opts AgentO
sw.Fail(stage, safeDuration(sw, agent.ReadyAt, agent.StartedAt))
// Use zero time (omitted) to separate these from the startup logs.
sw.Log(time.Time{}, codersdk.LogLevelWarn, "Warning: A startup script exited with an error and your workspace may be incomplete.")
sw.Log(time.Time{}, codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates#startup-script-exited-with-an-error"))
sw.Log(time.Time{}, codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates/troubleshooting#startup-script-exited-with-an-error"))
default:
switch {
case agent.LifecycleState.Starting():
// Use zero time (omitted) to separate these from the startup logs.
sw.Log(time.Time{}, codersdk.LogLevelWarn, "Notice: The startup scripts are still running and your workspace may be incomplete.")
sw.Log(time.Time{}, codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates#your-workspace-may-be-incomplete"))
sw.Log(time.Time{}, codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates/troubleshooting#your-workspace-may-be-incomplete"))
// Note: We don't complete or fail the stage here, it's
// intentionally left open to indicate this stage didn't
// complete.
@@ -236,7 +240,7 @@ func Agent(ctx context.Context, writer io.Writer, agentID uuid.UUID, opts AgentO
stage := "The workspace agent lost connection"
sw.Start(stage)
sw.Log(time.Now(), codersdk.LogLevelWarn, "Wait for it to reconnect or restart your workspace.")
sw.Log(time.Now(), codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates#agent-connection-issues"))
sw.Log(time.Now(), codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates/troubleshooting#agent-connection-issues"))
disconnectedAt := agent.DisconnectedAt
for agent.Status == codersdk.WorkspaceAgentDisconnected {
@@ -281,3 +285,55 @@ type closeFunc func() error
func (c closeFunc) Close() error {
return c()
}
func PeerDiagnostics(w io.Writer, d tailnet.PeerDiagnostics) {
if d.PreferredDERP > 0 {
rn, ok := d.DERPRegionNames[d.PreferredDERP]
if !ok {
rn = "unknown"
}
_, _ = fmt.Fprintf(w, "✔ preferred DERP region: %d (%s)\n", d.PreferredDERP, rn)
} else {
_, _ = fmt.Fprint(w, "✘ not connected to DERP\n")
}
if d.SentNode {
_, _ = fmt.Fprint(w, "✔ sent local data to Coder networking coodinator\n")
} else {
_, _ = fmt.Fprint(w, "✘ have not sent local data to Coder networking coordinator\n")
}
if d.ReceivedNode != nil {
dp := d.ReceivedNode.DERP
dn := ""
// should be 127.3.3.40:N where N is the DERP region
ap := strings.Split(dp, ":")
if len(ap) == 2 {
dp = ap[1]
di, err := strconv.Atoi(dp)
if err == nil {
var ok bool
dn, ok = d.DERPRegionNames[di]
if ok {
dn = fmt.Sprintf("(%s)", dn)
} else {
dn = "(unknown)"
}
}
}
_, _ = fmt.Fprintf(w,
"✔ received remote agent data from Coder networking coordinator\n preferred DERP region: %s %s\n endpoints: %s\n",
dp, dn, strings.Join(d.ReceivedNode.Endpoints, ", "))
} else {
_, _ = fmt.Fprint(w, "✘ have not received remote agent data from Coder networking coordinator\n")
}
if !d.LastWireguardHandshake.IsZero() {
ago := time.Since(d.LastWireguardHandshake)
symbol := "✔"
// wireguard is supposed to refresh handshake on 5 minute intervals
if ago > 5*time.Minute {
symbol = "⚠"
}
_, _ = fmt.Fprintf(w, "%s Wireguard handshake %s ago\n", symbol, ago.Round(time.Second))
} else {
_, _ = fmt.Fprint(w, "✘ Wireguard is not connected\n")
}
}
+196 -5
View File
@@ -6,6 +6,7 @@ import (
"context"
"io"
"os"
"regexp"
"strings"
"sync/atomic"
"testing"
@@ -15,13 +16,15 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/xerrors"
"tailscale.com/tailcfg"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/clitest"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/coderd/util/ptr"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/tailnet"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
func TestAgent(t *testing.T) {
@@ -379,8 +382,8 @@ func TestAgent(t *testing.T) {
output := make(chan string, 100) // Buffered to avoid blocking, overflow is discarded.
logs := make(chan []codersdk.WorkspaceAgentLog, 1)
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
tc.opts.Fetch = func(_ context.Context, _ uuid.UUID) (codersdk.WorkspaceAgent, error) {
t.Log("iter", len(tc.iter))
var err error
@@ -447,8 +450,8 @@ func TestAgent(t *testing.T) {
t.Parallel()
var fetchCalled uint64
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
buf := bytes.Buffer{}
err := cliui.Agent(inv.Context(), &buf, uuid.Nil, cliui.AgentOptions{
FetchInterval: 10 * time.Millisecond,
@@ -476,3 +479,191 @@ func TestAgent(t *testing.T) {
require.NoError(t, cmd.Invoke().Run())
})
}
func TestPeerDiagnostics(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
diags tailnet.PeerDiagnostics
want []*regexp.Regexp // must be ordered, can omit lines
}{
{
name: "noPreferredDERP",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: make(map[int]string),
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Now(),
},
want: []*regexp.Regexp{
regexp.MustCompile("^✘ not connected to DERP$"),
},
},
{
name: "preferredDERP",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 23,
DERPRegionNames: map[int]string{
23: "testo",
},
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Now(),
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ preferred DERP region: 23 \(testo\)$`),
},
},
{
name: "sentNode",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ sent local data to Coder networking coodinator$`),
},
},
{
name: "didntSendNode",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✘ have not sent local data to Coder networking coordinator$`),
},
},
{
name: "receivedNodeDERPOKNoEndpoints",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{999: "Embedded"},
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ received remote agent data from Coder networking coordinator$`),
regexp.MustCompile(`preferred DERP region: 999 \(Embedded\)$`),
regexp.MustCompile(`endpoints: $`),
},
},
{
name: "receivedNodeDERPUnknownNoEndpoints",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ received remote agent data from Coder networking coordinator$`),
regexp.MustCompile(`preferred DERP region: 999 \(unknown\)$`),
regexp.MustCompile(`endpoints: $`),
},
},
{
name: "receivedNodeEndpointsNoDERP",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{999: "Embedded"},
SentNode: true,
ReceivedNode: &tailcfg.Node{Endpoints: []string{"99.88.77.66:4555", "33.22.11.0:3444"}},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ received remote agent data from Coder networking coordinator$`),
regexp.MustCompile(`preferred DERP region:\s*$`),
regexp.MustCompile(`endpoints: 99\.88\.77\.66:4555, 33\.22\.11\.0:3444$`),
},
},
{
name: "didntReceiveNode",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: nil,
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✘ have not received remote agent data from Coder networking coordinator$`),
},
},
{
name: "noWireguardHandshake",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: nil,
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✘ Wireguard is not connected$`),
},
},
{
name: "wireguardHandshakeRecent",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: nil,
LastWireguardHandshake: time.Now().Add(-5 * time.Second),
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ Wireguard handshake \d+s ago$`),
},
},
{
name: "wireguardHandshakeOld",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: nil,
LastWireguardHandshake: time.Now().Add(-450 * time.Second), // 7m30s
},
want: []*regexp.Regexp{
regexp.MustCompile(`^⚠ Wireguard handshake 7m\d+s ago$`),
},
},
}
for _, tc := range testCases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
r, w := io.Pipe()
go func() {
defer w.Close()
cliui.PeerDiagnostics(w, tc.diags)
}()
s := bufio.NewScanner(r)
i := 0
got := make([]string, 0)
for s.Scan() {
got = append(got, s.Text())
if i < len(tc.want) {
reg := tc.want[i]
if reg.Match(s.Bytes()) {
i++
}
}
}
if i < len(tc.want) {
t.Logf("failed to match regexp: %s\ngot:\n%s", tc.want[i].String(), strings.Join(got, "\n"))
t.FailNow()
}
})
}
}
+4 -4
View File
@@ -3,13 +3,13 @@ package cliui
import (
"fmt"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/pretty"
"github.com/coder/serpent"
)
func DeprecationWarning(message string) clibase.MiddlewareFunc {
return func(next clibase.HandlerFunc) clibase.HandlerFunc {
return func(i *clibase.Invocation) error {
func DeprecationWarning(message string) serpent.MiddlewareFunc {
return func(next serpent.HandlerFunc) serpent.HandlerFunc {
return func(i *serpent.Invocation) error {
_, _ = fmt.Fprintln(i.Stdout, "\n"+pretty.Sprint(DefaultStyles.Wrap,
pretty.Sprint(
DefaultStyles.Warn,
+3 -3
View File
@@ -8,11 +8,11 @@ import (
"github.com/stretchr/testify/assert"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
func TestExternalAuth(t *testing.T) {
@@ -22,8 +22,8 @@ func TestExternalAuth(t *testing.T) {
defer cancel()
ptty := ptytest.New(t)
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
var fetched atomic.Bool
return cliui.ExternalAuth(inv.Context(), inv.Stdout, cliui.ExternalAuthOptions{
Fetch: func(ctx context.Context) ([]codersdk.TemplateVersionExternalAuth, error) {
+8 -8
View File
@@ -1,8 +1,8 @@
package cliui
import (
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
var defaultQuery = "owner:me"
@@ -11,12 +11,12 @@ var defaultQuery = "owner:me"
// and allows easy integration to a CLI command.
// Example usage:
//
// func (r *RootCmd) MyCmd() *clibase.Cmd {
// func (r *RootCmd) MyCmd() *serpent.Command {
// var (
// filter cliui.WorkspaceFilter
// ...
// )
// cmd := &clibase.Cmd{
// cmd := &serpent.Command{
// ...
// }
// filter.AttachOptions(&cmd.Options)
@@ -44,20 +44,20 @@ func (w *WorkspaceFilter) Filter() codersdk.WorkspaceFilter {
return f
}
func (w *WorkspaceFilter) AttachOptions(opts *clibase.OptionSet) {
func (w *WorkspaceFilter) AttachOptions(opts *serpent.OptionSet) {
*opts = append(*opts,
clibase.Option{
serpent.Option{
Flag: "all",
FlagShorthand: "a",
Description: "Specifies whether all workspaces will be listed or not.",
Value: clibase.BoolOf(&w.all),
Value: serpent.BoolOf(&w.all),
},
clibase.Option{
serpent.Option{
Flag: "search",
Description: "Search for a workspace with a query.",
Default: defaultQuery,
Value: clibase.StringOf(&w.searchQuery),
Value: serpent.StringOf(&w.searchQuery),
},
)
}
+12 -12
View File
@@ -9,12 +9,12 @@ import (
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/serpent"
)
type OutputFormat interface {
ID() string
AttachOptions(opts *clibase.OptionSet)
AttachOptions(opts *serpent.OptionSet)
Format(ctx context.Context, data any) (string, error)
}
@@ -49,7 +49,7 @@ func NewOutputFormatter(formats ...OutputFormat) *OutputFormatter {
// AttachOptions attaches the --output flag to the given command, and any
// additional flags required by the output formatters.
func (f *OutputFormatter) AttachOptions(opts *clibase.OptionSet) {
func (f *OutputFormatter) AttachOptions(opts *serpent.OptionSet) {
for _, format := range f.formats {
format.AttachOptions(opts)
}
@@ -60,11 +60,11 @@ func (f *OutputFormatter) AttachOptions(opts *clibase.OptionSet) {
}
*opts = append(*opts,
clibase.Option{
serpent.Option{
Flag: "output",
FlagShorthand: "o",
Default: f.formats[0].ID(),
Value: clibase.StringOf(&f.formatID),
Value: serpent.StringOf(&f.formatID),
Description: "Output format. Available formats: " + strings.Join(formatNames, ", ") + ".",
},
)
@@ -106,7 +106,7 @@ func TableFormat(out any, defaultColumns []string) OutputFormat {
}
// Get the list of table column headers.
headers, defaultSort, err := typeToTableHeaders(v.Type().Elem())
headers, defaultSort, err := typeToTableHeaders(v.Type().Elem(), true)
if err != nil {
panic("parse table headers: " + err.Error())
}
@@ -129,13 +129,13 @@ func (*tableFormat) ID() string {
}
// AttachOptions implements OutputFormat.
func (f *tableFormat) AttachOptions(opts *clibase.OptionSet) {
func (f *tableFormat) AttachOptions(opts *serpent.OptionSet) {
*opts = append(*opts,
clibase.Option{
serpent.Option{
Flag: "column",
FlagShorthand: "c",
Default: strings.Join(f.defaultColumns, ","),
Value: clibase.StringArrayOf(&f.columns),
Value: serpent.StringArrayOf(&f.columns),
Description: "Columns to display in table output. Available columns: " + strings.Join(f.allColumns, ", ") + ".",
},
)
@@ -161,7 +161,7 @@ func (jsonFormat) ID() string {
}
// AttachOptions implements OutputFormat.
func (jsonFormat) AttachOptions(_ *clibase.OptionSet) {}
func (jsonFormat) AttachOptions(_ *serpent.OptionSet) {}
// Format implements OutputFormat.
func (jsonFormat) Format(_ context.Context, data any) (string, error) {
@@ -187,7 +187,7 @@ func (textFormat) ID() string {
return "text"
}
func (textFormat) AttachOptions(_ *clibase.OptionSet) {}
func (textFormat) AttachOptions(_ *serpent.OptionSet) {}
func (textFormat) Format(_ context.Context, data any) (string, error) {
return fmt.Sprintf("%s", data), nil
@@ -213,7 +213,7 @@ func (d *DataChangeFormat) ID() string {
return d.format.ID()
}
func (d *DataChangeFormat) AttachOptions(opts *clibase.OptionSet) {
func (d *DataChangeFormat) AttachOptions(opts *serpent.OptionSet) {
d.format.AttachOptions(opts)
}
+7 -7
View File
@@ -8,13 +8,13 @@ import (
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/serpent"
)
type format struct {
id string
attachOptionsFn func(opts *clibase.OptionSet)
attachOptionsFn func(opts *serpent.OptionSet)
formatFn func(ctx context.Context, data any) (string, error)
}
@@ -24,7 +24,7 @@ func (f *format) ID() string {
return f.id
}
func (f *format) AttachOptions(opts *clibase.OptionSet) {
func (f *format) AttachOptions(opts *serpent.OptionSet) {
if f.attachOptionsFn != nil {
f.attachOptionsFn(opts)
}
@@ -85,12 +85,12 @@ func Test_OutputFormatter(t *testing.T) {
cliui.JSONFormat(),
&format{
id: "foo",
attachOptionsFn: func(opts *clibase.OptionSet) {
opts.Add(clibase.Option{
attachOptionsFn: func(opts *serpent.OptionSet) {
opts.Add(serpent.Option{
Name: "foo",
Flag: "foo",
FlagShorthand: "f",
Value: clibase.DiscardValue,
Value: serpent.DiscardValue,
Description: "foo flag 1234",
})
},
@@ -101,7 +101,7 @@ func Test_OutputFormatter(t *testing.T) {
},
)
cmd := &clibase.Cmd{}
cmd := &serpent.Command{}
f.AttachOptions(&cmd.Options)
fs := cmd.Options.FlagSet()
+2 -2
View File
@@ -5,12 +5,12 @@ import (
"fmt"
"strings"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/pretty"
"github.com/coder/serpent"
)
func RichParameter(inv *clibase.Invocation, templateVersionParameter codersdk.TemplateVersionParameter) (string, error) {
func RichParameter(inv *serpent.Invocation, templateVersionParameter codersdk.TemplateVersionParameter) (string, error) {
label := templateVersionParameter.Name
if templateVersionParameter.DisplayName != "" {
label = templateVersionParameter.DisplayName
+7 -7
View File
@@ -13,8 +13,8 @@ import (
"github.com/mattn/go-isatty"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/pretty"
"github.com/coder/serpent"
)
// PromptOptions supply a set of options to the prompt.
@@ -30,13 +30,13 @@ const skipPromptFlag = "yes"
// SkipPromptOption adds a "--yes/-y" flag to the cmd that can be used to skip
// prompts.
func SkipPromptOption() clibase.Option {
return clibase.Option{
func SkipPromptOption() serpent.Option {
return serpent.Option{
Flag: skipPromptFlag,
FlagShorthand: "y",
Description: "Bypass prompts.",
// Discard
Value: clibase.BoolOf(new(bool)),
Value: serpent.BoolOf(new(bool)),
}
}
@@ -46,7 +46,7 @@ const (
)
// Prompt asks the user for input.
func Prompt(inv *clibase.Invocation, opts PromptOptions) (string, error) {
func Prompt(inv *serpent.Invocation, opts PromptOptions) (string, error) {
// If the cmd has a "yes" flag for skipping confirm prompts, honor it.
// If it's not a "Confirm" prompt, then don't skip. As the default value of
// "yes" makes no sense.
@@ -71,9 +71,9 @@ func Prompt(inv *clibase.Invocation, opts PromptOptions) (string, error) {
} else {
renderedNo = Bold(ConfirmNo)
}
pretty.Fprintf(inv.Stdout, DefaultStyles.Placeholder, "(%s/%s) ", renderedYes, renderedNo)
_, _ = fmt.Fprintf(inv.Stdout, "(%s/%s) ", renderedYes, renderedNo)
} else if opts.Default != "" {
_, _ = fmt.Fprint(inv.Stdout, pretty.Sprint(DefaultStyles.Placeholder, "("+opts.Default+") "))
_, _ = fmt.Fprintf(inv.Stdout, "(%s) ", pretty.Sprint(DefaultStyles.Placeholder, opts.Default))
}
interrupt := make(chan os.Signal, 1)
+7 -7
View File
@@ -11,11 +11,11 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/pty"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
func TestPrompt(t *testing.T) {
@@ -77,7 +77,7 @@ func TestPrompt(t *testing.T) {
resp, err := newPrompt(ptty, cliui.PromptOptions{
Text: "ShouldNotSeeThis",
IsConfirm: true,
}, func(inv *clibase.Invocation) {
}, func(inv *serpent.Invocation) {
inv.Command.Options = append(inv.Command.Options, cliui.SkipPromptOption())
inv.Args = []string{"-y"}
})
@@ -145,10 +145,10 @@ func TestPrompt(t *testing.T) {
})
}
func newPrompt(ptty *ptytest.PTY, opts cliui.PromptOptions, invOpt func(inv *clibase.Invocation)) (string, error) {
func newPrompt(ptty *ptytest.PTY, opts cliui.PromptOptions, invOpt func(inv *serpent.Invocation)) (string, error) {
value := ""
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
var err error
value, err = cliui.Prompt(inv, opts)
return err
@@ -210,8 +210,8 @@ func TestPasswordTerminalState(t *testing.T) {
// nolint:unused
func passwordHelper() {
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
cliui.Prompt(inv, cliui.PromptOptions{
Text: "Password:",
Secret: true,
+35 -5
View File
@@ -54,6 +54,11 @@ func (err *ProvisionerJobError) Error() string {
return err.Message
}
const (
ProvisioningStateQueued = "Queued"
ProvisioningStateRunning = "Running"
)
// ProvisionerJob renders a provisioner job with interactive cancellation.
func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOptions) error {
if opts.FetchInterval == 0 {
@@ -63,8 +68,9 @@ func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOption
defer cancelFunc()
var (
currentStage = "Queued"
currentStage = ProvisioningStateQueued
currentStageStartedAt = time.Now().UTC()
currentQueuePos = -1
errChan = make(chan error, 1)
job codersdk.ProvisionerJob
@@ -74,7 +80,20 @@ func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOption
sw := &stageWriter{w: wr, verbose: opts.Verbose, silentLogs: opts.Silent}
printStage := func() {
sw.Start(currentStage)
out := currentStage
if currentStage == ProvisioningStateQueued && currentQueuePos > 0 {
var queuePos string
if currentQueuePos == 1 {
queuePos = "next"
} else {
queuePos = fmt.Sprintf("position: %d", currentQueuePos)
}
out = pretty.Sprintf(DefaultStyles.Warn, "%s (%s)", currentStage, queuePos)
}
sw.Start(out)
}
updateStage := func(stage string, startedAt time.Time) {
@@ -103,15 +122,26 @@ func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOption
errChan <- xerrors.Errorf("fetch: %w", err)
return
}
if job.QueuePosition != currentQueuePos {
initialState := currentQueuePos == -1
currentQueuePos = job.QueuePosition
// Print an update when the queue position changes, but:
// - not initially, because the stage is printed at startup
// - not when we're first in the queue, because it's redundant
if !initialState && currentQueuePos != 0 {
printStage()
}
}
if job.StartedAt == nil {
return
}
if currentStage != "Queued" {
if currentStage != ProvisioningStateQueued {
// If another stage is already running, there's no need
// for us to notify the user we're running!
return
}
updateStage("Running", *job.StartedAt)
updateStage(ProvisioningStateRunning, *job.StartedAt)
}
if opts.Cancel != nil {
@@ -143,8 +173,8 @@ func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOption
}
// The initial stage needs to print after the signal handler has been registered.
printStage()
updateJob()
printStage()
logs, closer, err := opts.Logs()
if err != nil {
+120 -26
View File
@@ -2,8 +2,10 @@ package cliui_test
import (
"context"
"fmt"
"io"
"os"
"regexp"
"runtime"
"sync"
"testing"
@@ -11,11 +13,13 @@ import (
"github.com/stretchr/testify/assert"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/testutil"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/coderd/database/dbtime"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/serpent"
)
// This cannot be ran in parallel because it uses a signal.
@@ -25,7 +29,11 @@ func TestProvisionerJob(t *testing.T) {
t.Parallel()
test := newProvisionerJob(t)
go func() {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancel()
testutil.Go(t, func() {
<-test.Next
test.JobMutex.Lock()
test.Job.Status = codersdk.ProvisionerJobRunning
@@ -39,20 +47,26 @@ func TestProvisionerJob(t *testing.T) {
test.Job.CompletedAt = &now
close(test.Logs)
test.JobMutex.Unlock()
}()
test.PTY.ExpectMatch("Queued")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Queued")
test.PTY.ExpectMatch("Running")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Running")
})
testutil.Eventually(ctx, t, func(ctx context.Context) (done bool) {
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.PTY.ExpectMatch(cliui.ProvisioningStateRunning)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateRunning)
return true
}, testutil.IntervalFast)
})
t.Run("Stages", func(t *testing.T) {
t.Parallel()
test := newProvisionerJob(t)
go func() {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancel()
testutil.Go(t, func() {
<-test.Next
test.JobMutex.Lock()
test.Job.Status = codersdk.ProvisionerJobRunning
@@ -70,13 +84,86 @@ func TestProvisionerJob(t *testing.T) {
test.Job.CompletedAt = &now
close(test.Logs)
test.JobMutex.Unlock()
}()
test.PTY.ExpectMatch("Queued")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Queued")
test.PTY.ExpectMatch("Something")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Something")
})
testutil.Eventually(ctx, t, func(ctx context.Context) (done bool) {
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.PTY.ExpectMatch("Something")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Something")
return true
}, testutil.IntervalFast)
})
t.Run("Queue Position", func(t *testing.T) {
t.Parallel()
stage := cliui.ProvisioningStateQueued
tests := []struct {
name string
queuePos int
expected string
}{
{
name: "first",
queuePos: 0,
expected: fmt.Sprintf("%s$", stage),
},
{
name: "next",
queuePos: 1,
expected: fmt.Sprintf(`%s %s$`, stage, regexp.QuoteMeta("(next)")),
},
{
name: "other",
queuePos: 4,
expected: fmt.Sprintf(`%s %s$`, stage, regexp.QuoteMeta("(position: 4)")),
},
}
for _, tc := range tests {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
test := newProvisionerJob(t)
test.JobMutex.Lock()
test.Job.QueuePosition = tc.queuePos
test.Job.QueueSize = tc.queuePos
test.JobMutex.Unlock()
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancel()
testutil.Go(t, func() {
<-test.Next
test.JobMutex.Lock()
test.Job.Status = codersdk.ProvisionerJobRunning
now := dbtime.Now()
test.Job.StartedAt = &now
test.JobMutex.Unlock()
<-test.Next
test.JobMutex.Lock()
test.Job.Status = codersdk.ProvisionerJobSucceeded
now = dbtime.Now()
test.Job.CompletedAt = &now
close(test.Logs)
test.JobMutex.Unlock()
})
testutil.Eventually(ctx, t, func(ctx context.Context) (done bool) {
test.PTY.ExpectRegexMatch(tc.expected)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued) // step completed
test.PTY.ExpectMatch(cliui.ProvisioningStateRunning)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateRunning)
return true
}, testutil.IntervalFast)
})
}
})
// This cannot be ran in parallel because it uses a signal.
@@ -90,7 +177,11 @@ func TestProvisionerJob(t *testing.T) {
}
test := newProvisionerJob(t)
go func() {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancel()
testutil.Go(t, func() {
<-test.Next
currentProcess, err := os.FindProcess(os.Getpid())
assert.NoError(t, err)
@@ -103,12 +194,15 @@ func TestProvisionerJob(t *testing.T) {
test.Job.CompletedAt = &now
close(test.Logs)
test.JobMutex.Unlock()
}()
test.PTY.ExpectMatch("Queued")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Gracefully canceling")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Queued")
})
testutil.Eventually(ctx, t, func(ctx context.Context) (done bool) {
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.Next <- struct{}{}
test.PTY.ExpectMatch("Gracefully canceling")
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
return true
}, testutil.IntervalFast)
})
}
@@ -127,8 +221,8 @@ func newProvisionerJob(t *testing.T) provisionerJobTest {
}
jobLock := sync.Mutex{}
logs := make(chan codersdk.ProvisionerJobLog, 1)
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
return cliui.ProvisionerJob(inv.Context(), inv.Stdout, cliui.ProvisionerJobOptions{
FetchInterval: time.Millisecond,
Fetch: func() (codersdk.ProvisionerJob, error) {
+4 -4
View File
@@ -10,8 +10,8 @@ import (
"github.com/AlecAivazis/survey/v2/terminal"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
func init() {
@@ -68,7 +68,7 @@ type RichSelectOptions struct {
}
// RichSelect displays a list of user options including name and description.
func RichSelect(inv *clibase.Invocation, richOptions RichSelectOptions) (*codersdk.TemplateVersionParameterOption, error) {
func RichSelect(inv *serpent.Invocation, richOptions RichSelectOptions) (*codersdk.TemplateVersionParameterOption, error) {
opts := make([]string, len(richOptions.Options))
var defaultOpt string
for i, option := range richOptions.Options {
@@ -102,7 +102,7 @@ func RichSelect(inv *clibase.Invocation, richOptions RichSelectOptions) (*coders
}
// Select displays a list of user options.
func Select(inv *clibase.Invocation, opts SelectOptions) (string, error) {
func Select(inv *serpent.Invocation, opts SelectOptions) (string, error) {
// The survey library used *always* fails when testing on Windows,
// as it requires a live TTY (can't be a conpty). We should fork
// this library to add a dummy fallback, that simply reads/writes
@@ -138,7 +138,7 @@ func Select(inv *clibase.Invocation, opts SelectOptions) (string, error) {
return value, err
}
func MultiSelect(inv *clibase.Invocation, items []string) ([]string, error) {
func MultiSelect(inv *serpent.Invocation, items []string) ([]string, error) {
// Similar hack is applied to Select()
if flag.Lookup("test.v") != nil {
return items, nil
+7 -7
View File
@@ -6,10 +6,10 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/serpent"
)
func TestSelect(t *testing.T) {
@@ -31,8 +31,8 @@ func TestSelect(t *testing.T) {
func newSelect(ptty *ptytest.PTY, opts cliui.SelectOptions) (string, error) {
value := ""
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
var err error
value, err = cliui.Select(inv, opts)
return err
@@ -72,8 +72,8 @@ func TestRichSelect(t *testing.T) {
func newRichSelect(ptty *ptytest.PTY, opts cliui.RichSelectOptions) (string, error) {
value := ""
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
richOption, err := cliui.RichSelect(inv, opts)
if err == nil {
value = richOption.Value
@@ -105,8 +105,8 @@ func TestMultiSelect(t *testing.T) {
func newMultiSelect(ptty *ptytest.PTY, items []string) ([]string, error) {
var values []string
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
selectedItems, err := cliui.MultiSelect(inv, items)
if err == nil {
values = selectedItems
+17 -4
View File
@@ -70,7 +70,7 @@ func DisplayTable(out any, sort string, filterColumns []string) (string, error)
}
// Get the list of table column headers.
headersRaw, defaultSort, err := typeToTableHeaders(v.Type().Elem())
headersRaw, defaultSort, err := typeToTableHeaders(v.Type().Elem(), true)
if err != nil {
return "", xerrors.Errorf("get table headers recursively for type %q: %w", v.Type().Elem().String(), err)
}
@@ -230,7 +230,11 @@ func isStructOrStructPointer(t reflect.Type) bool {
// typeToTableHeaders converts a type to a slice of column names. If the given
// type is invalid (not a struct or a pointer to a struct, has invalid table
// tags, etc.), an error is returned.
func typeToTableHeaders(t reflect.Type) ([]string, string, error) {
//
// requireDefault is only needed for the root call. This is recursive, so nested
// structs do not need the default sort name.
// nolint:revive
func typeToTableHeaders(t reflect.Type, requireDefault bool) ([]string, string, error) {
if !isStructOrStructPointer(t) {
return nil, "", xerrors.Errorf("typeToTableHeaders called with a non-struct or a non-pointer-to-a-struct type")
}
@@ -246,6 +250,12 @@ func typeToTableHeaders(t reflect.Type) ([]string, string, error) {
if err != nil {
return nil, "", xerrors.Errorf("parse struct tags for field %q in type %q: %w", field.Name, t.String(), err)
}
if name == "" && (recursive && skip) {
return nil, "", xerrors.Errorf("a name is required for the field %q. "+
"recursive_line will ensure this is never shown to the user, but is still needed", field.Name)
}
// If recurse and skip is set, the name is intentionally empty.
if name == "" {
continue
}
@@ -262,7 +272,7 @@ func typeToTableHeaders(t reflect.Type) ([]string, string, error) {
return nil, "", xerrors.Errorf("field %q in type %q is marked as recursive but does not contain a struct or a pointer to a struct", field.Name, t.String())
}
childNames, _, err := typeToTableHeaders(fieldType)
childNames, defaultSort, err := typeToTableHeaders(fieldType, false)
if err != nil {
return nil, "", xerrors.Errorf("get child field header names for field %q in type %q: %w", field.Name, fieldType.String(), err)
}
@@ -273,13 +283,16 @@ func typeToTableHeaders(t reflect.Type) ([]string, string, error) {
}
headers = append(headers, fullName)
}
if defaultSortName == "" {
defaultSortName = defaultSort
}
continue
}
headers = append(headers, name)
}
if defaultSortName == "" {
if defaultSortName == "" && requireDefault {
return nil, "", xerrors.Errorf("no field marked as default_sort in type %q", t.String())
}
+2 -2
View File
@@ -46,12 +46,12 @@ type tableTest2 struct {
type tableTest3 struct {
NotIncluded string // no table tag
Sub tableTest2 `table:"inner,recursive,default_sort"`
Sub tableTest2 `table:"inner,recursive"`
}
type tableTest4 struct {
Inline tableTest2 `table:"ignored,recursive_inline"`
SortField string `table:"sort_field,default_sort"`
SortField string `table:"sort_field"`
}
func Test_DisplayTable(t *testing.T) {
+12 -2
View File
@@ -4,6 +4,7 @@ import (
"io"
"os"
"path/filepath"
"strings"
"github.com/kirsle/configdir"
"golang.org/x/xerrors"
@@ -69,6 +70,14 @@ func (r Root) PostgresPort() File {
// File provides convenience methods for interacting with *os.File.
type File string
func (f File) Exists() bool {
if f == "" {
return false
}
_, err := os.Stat(string(f))
return err == nil
}
// Delete deletes the file.
func (f File) Delete() error {
if f == "" {
@@ -85,13 +94,14 @@ func (f File) Write(s string) error {
return write(string(f), 0o600, []byte(s))
}
// Read reads the file to a string.
// Read reads the file to a string. All leading and trailing whitespace
// is removed.
func (f File) Read() (string, error) {
if f == "" {
return "", xerrors.Errorf("empty file path")
}
byt, err := read(string(f))
return string(byt), err
return strings.TrimSpace(string(byt)), err
}
// open opens a file in the configuration directory,
+66 -26
View File
@@ -19,14 +19,15 @@ import (
"github.com/cli/safeexec"
"github.com/pkg/diff"
"github.com/pkg/diff/write"
"golang.org/x/exp/constraints"
"golang.org/x/exp/slices"
"golang.org/x/sync/errgroup"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/coderd/util/slice"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
const (
@@ -51,6 +52,8 @@ type sshConfigOptions struct {
userHostPrefix string
sshOptions []string
disableAutostart bool
header []string
headerCommand string
}
// addOptions expects options in the form of "option=value" or "option value".
@@ -100,15 +103,25 @@ func (o *sshConfigOptions) addOption(option string) error {
}
func (o sshConfigOptions) equal(other sshConfigOptions) bool {
// Compare without side-effects or regard to order.
opt1 := slices.Clone(o.sshOptions)
sort.Strings(opt1)
opt2 := slices.Clone(other.sshOptions)
sort.Strings(opt2)
if !slices.Equal(opt1, opt2) {
if !slicesSortedEqual(o.sshOptions, other.sshOptions) {
return false
}
return o.waitEnum == other.waitEnum && o.userHostPrefix == other.userHostPrefix && o.disableAutostart == other.disableAutostart
if !slicesSortedEqual(o.header, other.header) {
return false
}
return o.waitEnum == other.waitEnum && o.userHostPrefix == other.userHostPrefix && o.disableAutostart == other.disableAutostart && o.headerCommand == other.headerCommand
}
// slicesSortedEqual compares two slices without side-effects or regard to order.
func slicesSortedEqual[S ~[]E, E constraints.Ordered](a, b S) bool {
if len(a) != len(b) {
return false
}
a = slices.Clone(a)
slices.Sort(a)
b = slices.Clone(b)
slices.Sort(b)
return slices.Equal(a, b)
}
func (o sshConfigOptions) asList() (list []string) {
@@ -124,6 +137,13 @@ func (o sshConfigOptions) asList() (list []string) {
for _, opt := range o.sshOptions {
list = append(list, fmt.Sprintf("ssh-option: %s", opt))
}
for _, h := range o.header {
list = append(list, fmt.Sprintf("header: %s", h))
}
if o.headerCommand != "" {
list = append(list, fmt.Sprintf("header-command: %s", o.headerCommand))
}
return list
}
@@ -195,7 +215,7 @@ func sshPrepareWorkspaceConfigs(ctx context.Context, client *codersdk.Client) (r
}
}
func (r *RootCmd) configSSH() *clibase.Cmd {
func (r *RootCmd) configSSH() *serpent.Command {
var (
sshConfigFile string
sshConfigOpts sshConfigOptions
@@ -206,7 +226,7 @@ func (r *RootCmd) configSSH() *clibase.Cmd {
coderCliPath string
)
client := new(codersdk.Client)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Annotations: workspaceCommand,
Use: "config-ssh",
Short: "Add an SSH Host entry for your workspaces \"ssh coder.workspace\"",
@@ -220,16 +240,18 @@ func (r *RootCmd) configSSH() *clibase.Cmd {
Command: "coder config-ssh --dry-run",
},
),
Middleware: clibase.Chain(
clibase.RequireNArgs(0),
Middleware: serpent.Chain(
serpent.RequireNArgs(0),
r.InitClient(client),
),
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
if sshConfigOpts.waitEnum != "auto" && skipProxyCommand {
// The wait option is applied to the ProxyCommand. If the user
// specifies skip-proxy-command, then wait cannot be applied.
return xerrors.Errorf("cannot specify both --skip-proxy-command and --wait")
}
sshConfigOpts.header = r.header
sshConfigOpts.headerCommand = r.headerCommand
recvWorkspaceConfigs := sshPrepareWorkspaceConfigs(inv.Context(), client)
@@ -393,6 +415,14 @@ func (r *RootCmd) configSSH() *clibase.Cmd {
}
if !skipProxyCommand {
rootFlags := fmt.Sprintf("--global-config %s", escapedGlobalConfig)
for _, h := range sshConfigOpts.header {
rootFlags += fmt.Sprintf(" --header %q", h)
}
if sshConfigOpts.headerCommand != "" {
rootFlags += fmt.Sprintf(" --header-command %q", sshConfigOpts.headerCommand)
}
flags := ""
if sshConfigOpts.waitEnum != "auto" {
flags += " --wait=" + sshConfigOpts.waitEnum
@@ -401,8 +431,8 @@ func (r *RootCmd) configSSH() *clibase.Cmd {
flags += " --disable-autostart=true"
}
defaultOptions = append(defaultOptions, fmt.Sprintf(
"ProxyCommand %s --global-config %s ssh --stdio%s %s",
escapedCoderBinary, escapedGlobalConfig, flags, workspaceHostname,
"ProxyCommand %s %s ssh --stdio%s %s",
escapedCoderBinary, rootFlags, flags, workspaceHostname,
))
}
@@ -508,13 +538,13 @@ func (r *RootCmd) configSSH() *clibase.Cmd {
},
}
cmd.Options = clibase.OptionSet{
cmd.Options = serpent.OptionSet{
{
Flag: "ssh-config-file",
Env: "CODER_SSH_CONFIG_FILE",
Default: sshDefaultConfigFileName,
Description: "Specifies the path to an SSH config.",
Value: clibase.StringOf(&sshConfigFile),
Value: serpent.StringOf(&sshConfigFile),
},
{
Flag: "coder-binary-path",
@@ -522,7 +552,7 @@ func (r *RootCmd) configSSH() *clibase.Cmd {
Default: "",
Description: "Optionally specify the absolute path to the coder binary used in ProxyCommand. " +
"By default, the binary invoking this command ('config ssh') is used.",
Value: clibase.Validate(clibase.StringOf(&coderCliPath), func(value *clibase.String) error {
Value: serpent.Validate(serpent.StringOf(&coderCliPath), func(value *serpent.String) error {
if runtime.GOOS == goosWindows {
// For some reason filepath.IsAbs() does not work on windows.
return nil
@@ -539,46 +569,46 @@ func (r *RootCmd) configSSH() *clibase.Cmd {
FlagShorthand: "o",
Env: "CODER_SSH_CONFIG_OPTS",
Description: "Specifies additional SSH options to embed in each host stanza.",
Value: clibase.StringArrayOf(&sshConfigOpts.sshOptions),
Value: serpent.StringArrayOf(&sshConfigOpts.sshOptions),
},
{
Flag: "dry-run",
FlagShorthand: "n",
Env: "CODER_SSH_DRY_RUN",
Description: "Perform a trial run with no changes made, showing a diff at the end.",
Value: clibase.BoolOf(&dryRun),
Value: serpent.BoolOf(&dryRun),
},
{
Flag: "skip-proxy-command",
Env: "CODER_SSH_SKIP_PROXY_COMMAND",
Description: "Specifies whether the ProxyCommand option should be skipped. Useful for testing.",
Value: clibase.BoolOf(&skipProxyCommand),
Value: serpent.BoolOf(&skipProxyCommand),
Hidden: true,
},
{
Flag: "use-previous-options",
Env: "CODER_SSH_USE_PREVIOUS_OPTIONS",
Description: "Specifies whether or not to keep options from previous run of config-ssh.",
Value: clibase.BoolOf(&usePreviousOpts),
Value: serpent.BoolOf(&usePreviousOpts),
},
{
Flag: "ssh-host-prefix",
Env: "CODER_CONFIGSSH_SSH_HOST_PREFIX",
Description: "Override the default host prefix.",
Value: clibase.StringOf(&sshConfigOpts.userHostPrefix),
Value: serpent.StringOf(&sshConfigOpts.userHostPrefix),
},
{
Flag: "wait",
Env: "CODER_CONFIGSSH_WAIT", // Not to be mixed with CODER_SSH_WAIT.
Description: "Specifies whether or not to wait for the startup script to finish executing. Auto means that the agent startup script behavior configured in the workspace template is used.",
Default: "auto",
Value: clibase.EnumOf(&sshConfigOpts.waitEnum, "yes", "no", "auto"),
Value: serpent.EnumOf(&sshConfigOpts.waitEnum, "yes", "no", "auto"),
},
{
Flag: "disable-autostart",
Description: "Disable starting the workspace automatically when connecting via SSH.",
Env: "CODER_CONFIGSSH_DISABLE_AUTOSTART",
Value: clibase.BoolOf(&sshConfigOpts.disableAutostart),
Value: serpent.BoolOf(&sshConfigOpts.disableAutostart),
Default: "false",
},
{
@@ -587,7 +617,7 @@ func (r *RootCmd) configSSH() *clibase.Cmd {
Description: "By default, 'config-ssh' uses the os path separator when writing the ssh config. " +
"This might be an issue in Windows machine that use a unix-like shell. " +
"This flag forces the use of unix file paths (the forward slash '/').",
Value: clibase.BoolOf(&forceUnixSeparators),
Value: serpent.BoolOf(&forceUnixSeparators),
// On non-windows showing this command is useless because it is a noop.
// Hide vs disable it though so if a command is copied from a Windows
// machine to a unix machine it will still work and not throw an
@@ -623,6 +653,12 @@ func sshConfigWriteSectionHeader(w io.Writer, addNewline bool, o sshConfigOption
for _, opt := range o.sshOptions {
_, _ = fmt.Fprintf(&ow, "# :%s=%s\n", "ssh-option", opt)
}
for _, h := range o.header {
_, _ = fmt.Fprintf(&ow, "# :%s=%s\n", "header", h)
}
if o.headerCommand != "" {
_, _ = fmt.Fprintf(&ow, "# :%s=%s\n", "header-command", o.headerCommand)
}
if ow.Len() > 0 {
_, _ = fmt.Fprint(w, sshConfigOptionsHeader)
_, _ = fmt.Fprint(w, ow.String())
@@ -654,6 +690,10 @@ func sshConfigParseLastOptions(r io.Reader) (o sshConfigOptions) {
o.sshOptions = append(o.sshOptions, parts[1])
case "disable-autostart":
o.disableAutostart, _ = strconv.ParseBool(parts[1])
case "header":
o.header = append(o.header, parts[1])
case "header-command":
o.headerCommand = parts[1]
default:
// Unknown option, ignore.
}
+58 -1
View File
@@ -25,6 +25,7 @@ import (
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/dbfake"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
"github.com/coder/coder/v2/provisionersdk/proto"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/coder/v2/testutil"
@@ -83,7 +84,8 @@ func TestConfigSSH(t *testing.T) {
}).WithAgent().Do()
_ = agenttest.New(t, client.URL, r.AgentToken)
resources := coderdtest.AwaitWorkspaceAgents(t, client, r.Workspace.ID)
agentConn, err := client.DialWorkspaceAgent(context.Background(), resources[0].Agents[0].ID, nil)
agentConn, err := workspacesdk.New(client).
DialAgent(context.Background(), resources[0].Agents[0].ID, nil)
require.NoError(t, err)
defer agentConn.Close()
@@ -462,6 +464,9 @@ func TestConfigSSH_FileWriteAndOptionsFlow(t *testing.T) {
"# Last config-ssh options:",
"# :wait=yes",
"# :ssh-host-prefix=coder-test.",
"# :header=X-Test-Header=foo",
"# :header=X-Test-Header2=bar",
"# :header-command=printf h1=v1 h2=\"v2\" h3='v3'",
"#",
headerEnd,
"",
@@ -471,6 +476,9 @@ func TestConfigSSH_FileWriteAndOptionsFlow(t *testing.T) {
"--yes",
"--wait=yes",
"--ssh-host-prefix", "coder-test.",
"--header", "X-Test-Header=foo",
"--header", "X-Test-Header2=bar",
"--header-command", "printf h1=v1 h2=\"v2\" h3='v3'",
},
},
{
@@ -563,6 +571,55 @@ func TestConfigSSH_FileWriteAndOptionsFlow(t *testing.T) {
regexMatch: "ProxyCommand /foo/bar/coder",
},
},
{
name: "Header",
args: []string{
"--yes",
"--header", "X-Test-Header=foo",
"--header", "X-Test-Header2=bar",
},
wantErr: false,
hasAgent: true,
wantConfig: wantConfig{
regexMatch: `ProxyCommand .* --header "X-Test-Header=foo" --header "X-Test-Header2=bar" ssh`,
},
},
{
name: "Header command",
args: []string{
"--yes",
"--header-command", "printf h1=v1",
},
wantErr: false,
hasAgent: true,
wantConfig: wantConfig{
regexMatch: `ProxyCommand .* --header-command "printf h1=v1" ssh`,
},
},
{
name: "Header command with double quotes",
args: []string{
"--yes",
"--header-command", "printf h1=v1 h2=\"v2\"",
},
wantErr: false,
hasAgent: true,
wantConfig: wantConfig{
regexMatch: `ProxyCommand .* --header-command "printf h1=v1 h2=\\\"v2\\\"" ssh`,
},
},
{
name: "Header command with single quotes",
args: []string{
"--yes",
"--header-command", "printf h1=v1 h2='v2'",
},
wantErr: false,
hasAgent: true,
wantConfig: wantConfig{
regexMatch: `ProxyCommand .* --header-command "printf h1=v1 h2='v2'" ssh`,
},
},
}
for _, tt := range tests {
tt := tt
+17 -17
View File
@@ -12,14 +12,14 @@ import (
"github.com/coder/pretty"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/coderd/util/ptr"
"github.com/coder/coder/v2/coderd/util/slice"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
func (r *RootCmd) create() *clibase.Cmd {
func (r *RootCmd) create() *serpent.Command {
var (
templateName string
startAt string
@@ -31,7 +31,7 @@ func (r *RootCmd) create() *clibase.Cmd {
copyParametersFrom string
)
client := new(codersdk.Client)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Annotations: workspaceCommand,
Use: "create [name]",
Short: "Create a workspace",
@@ -41,9 +41,9 @@ func (r *RootCmd) create() *clibase.Cmd {
Command: "coder create <username>/<workspace_name>",
},
),
Middleware: clibase.Chain(r.InitClient(client)),
Handler: func(inv *clibase.Invocation) error {
organization, err := CurrentOrganization(inv, client)
Middleware: serpent.Chain(r.InitClient(client)),
Handler: func(inv *serpent.Invocation) error {
organization, err := CurrentOrganization(r, inv, client)
if err != nil {
return err
}
@@ -227,37 +227,37 @@ func (r *RootCmd) create() *clibase.Cmd {
},
}
cmd.Options = append(cmd.Options,
clibase.Option{
serpent.Option{
Flag: "template",
FlagShorthand: "t",
Env: "CODER_TEMPLATE_NAME",
Description: "Specify a template name.",
Value: clibase.StringOf(&templateName),
Value: serpent.StringOf(&templateName),
},
clibase.Option{
serpent.Option{
Flag: "start-at",
Env: "CODER_WORKSPACE_START_AT",
Description: "Specify the workspace autostart schedule. Check coder schedule start --help for the syntax.",
Value: clibase.StringOf(&startAt),
Value: serpent.StringOf(&startAt),
},
clibase.Option{
serpent.Option{
Flag: "stop-after",
Env: "CODER_WORKSPACE_STOP_AFTER",
Description: "Specify a duration after which the workspace should shut down (e.g. 8h).",
Value: clibase.DurationOf(&stopAfter),
Value: serpent.DurationOf(&stopAfter),
},
clibase.Option{
serpent.Option{
Flag: "automatic-updates",
Env: "CODER_WORKSPACE_AUTOMATIC_UPDATES",
Description: "Specify automatic updates setting for the workspace (accepts 'always' or 'never').",
Default: string(codersdk.AutomaticUpdatesNever),
Value: clibase.StringOf(&autoUpdates),
Value: serpent.StringOf(&autoUpdates),
},
clibase.Option{
serpent.Option{
Flag: "copy-parameters-from",
Env: "CODER_WORKSPACE_COPY_PARAMETERS_FROM",
Description: "Specify the source workspace name to copy parameters from.",
Value: clibase.StringOf(&copyParametersFrom),
Value: serpent.StringOf(&copyParametersFrom),
},
cliui.SkipPromptOption(),
)
@@ -283,7 +283,7 @@ type prepWorkspaceBuildArgs struct {
// prepWorkspaceBuild will ensure a workspace build will succeed on the latest template version.
// Any missing params will be prompted to the user. It supports rich parameters.
func prepWorkspaceBuild(inv *clibase.Invocation, client *codersdk.Client, args prepWorkspaceBuildArgs) ([]codersdk.WorkspaceBuildParameter, error) {
func prepWorkspaceBuild(inv *serpent.Invocation, client *codersdk.Client, args prepWorkspaceBuildArgs) ([]codersdk.WorkspaceBuildParameter, error) {
ctx := inv.Context()
templateVersion, err := client.TemplateVersion(ctx, args.TemplateVersionID)
+47 -1
View File
@@ -556,6 +556,14 @@ func TestCreateValidateRichParameters(t *testing.T) {
{Name: numberParameterName, Type: "number", Mutable: true, ValidationMin: ptr.Ref(int32(3)), ValidationMax: ptr.Ref(int32(10))},
}
numberCustomErrorRichParameters := []*proto.RichParameter{
{
Name: numberParameterName, Type: "number", Mutable: true,
ValidationMin: ptr.Ref(int32(3)), ValidationMax: ptr.Ref(int32(10)),
ValidationError: "These are values: {min}, {max}, and {value}.",
},
}
stringRichParameters := []*proto.RichParameter{
{Name: stringParameterName, Type: "string", Mutable: true, ValidationRegex: "^[a-z]+$", ValidationError: "this is error"},
}
@@ -644,6 +652,44 @@ func TestCreateValidateRichParameters(t *testing.T) {
<-doneChan
})
t.Run("ValidateNumber_CustomError", func(t *testing.T) {
t.Parallel()
client := coderdtest.New(t, &coderdtest.Options{IncludeProvisionerDaemon: true})
owner := coderdtest.CreateFirstUser(t, client)
member, _ := coderdtest.CreateAnotherUser(t, client, owner.OrganizationID)
version := coderdtest.CreateTemplateVersion(t, client, owner.OrganizationID, prepareEchoResponses(numberCustomErrorRichParameters))
coderdtest.AwaitTemplateVersionJobCompleted(t, client, version.ID)
template := coderdtest.CreateTemplate(t, client, owner.OrganizationID, version.ID)
inv, root := clitest.New(t, "create", "my-workspace", "--template", template.Name)
clitest.SetupConfig(t, member, root)
doneChan := make(chan struct{})
pty := ptytest.New(t).Attach(inv)
go func() {
defer close(doneChan)
err := inv.Run()
assert.NoError(t, err)
}()
matches := []string{
numberParameterName, "12",
"These are values: 3, 10, and 12.", "",
"Enter a value", "8",
"Confirm create?", "yes",
}
for i := 0; i < len(matches); i += 2 {
match := matches[i]
value := matches[i+1]
pty.ExpectMatch(match)
if value != "" {
pty.WriteLine(value)
}
}
<-doneChan
})
t.Run("ValidateBool", func(t *testing.T) {
t.Parallel()
@@ -757,7 +803,7 @@ func TestCreateWithGitAuth(t *testing.T) {
{
Type: &proto.Response_Plan{
Plan: &proto.PlanComplete{
ExternalAuthProviders: []string{"github"},
ExternalAuthProviders: []*proto.ExternalAuthProviderResource{{Id: "github"}},
},
},
},
+8 -8
View File
@@ -4,24 +4,24 @@ import (
"fmt"
"time"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
// nolint
func (r *RootCmd) deleteWorkspace() *clibase.Cmd {
func (r *RootCmd) deleteWorkspace() *serpent.Command {
var orphan bool
client := new(codersdk.Client)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Annotations: workspaceCommand,
Use: "delete <workspace>",
Short: "Delete a workspace",
Middleware: clibase.Chain(
clibase.RequireNArgs(1),
Middleware: serpent.Chain(
serpent.RequireNArgs(1),
r.InitClient(client),
),
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
workspace, err := namedWorkspace(inv.Context(), client, inv.Args[0])
if err != nil {
return err
@@ -62,12 +62,12 @@ func (r *RootCmd) deleteWorkspace() *clibase.Cmd {
return nil
},
}
cmd.Options = clibase.OptionSet{
cmd.Options = serpent.OptionSet{
{
Flag: "orphan",
Description: "Delete a workspace without deleting its resources. This can delete a workspace in a broken state, but may also lead to unaccounted cloud resources.",
Value: clibase.BoolOf(&orphan),
Value: serpent.BoolOf(&orphan),
},
cliui.SkipPromptOption(),
}
+1 -5
View File
@@ -11,7 +11,6 @@ import (
"github.com/coder/coder/v2/cli/clitest"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/dbauthz"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/pty/ptytest"
@@ -95,10 +94,7 @@ func TestDelete(t *testing.T) {
// this way.
ctx := testutil.Context(t, testutil.WaitShort)
// nolint:gocritic // Unit test
err := api.Database.UpdateUserDeletedByID(dbauthz.AsSystemRestricted(ctx), database.UpdateUserDeletedByIDParams{
ID: deleteMeUser.ID,
Deleted: true,
})
err := api.Database.UpdateUserDeletedByID(dbauthz.AsSystemRestricted(ctx), deleteMeUser.ID)
require.NoError(t, err)
inv, root := clitest.New(t, "delete", fmt.Sprintf("%s/%s", deleteMeUser.ID, workspace.Name), "-y", "--orphan")
+10 -10
View File
@@ -15,18 +15,18 @@ import (
"github.com/coder/pretty"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/serpent"
)
func (r *RootCmd) dotfiles() *clibase.Cmd {
func (r *RootCmd) dotfiles() *serpent.Command {
var symlinkDir string
var gitbranch string
var dotfilesRepoDir string
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "dotfiles <git_repo_url>",
Middleware: clibase.RequireNArgs(1),
Middleware: serpent.RequireNArgs(1),
Short: "Personalize your workspace by applying a canonical dotfiles repository",
Long: formatExamples(
example{
@@ -34,7 +34,7 @@ func (r *RootCmd) dotfiles() *clibase.Cmd {
Command: "coder dotfiles --yes git@github.com:example/dotfiles.git",
},
),
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
var (
gitRepo = inv.Args[0]
cfg = r.createConfig()
@@ -276,26 +276,26 @@ func (r *RootCmd) dotfiles() *clibase.Cmd {
return nil
},
}
cmd.Options = clibase.OptionSet{
cmd.Options = serpent.OptionSet{
{
Flag: "symlink-dir",
Env: "CODER_SYMLINK_DIR",
Description: "Specifies the directory for the dotfiles symlink destinations. If empty, will use $HOME.",
Value: clibase.StringOf(&symlinkDir),
Value: serpent.StringOf(&symlinkDir),
},
{
Flag: "branch",
FlagShorthand: "b",
Description: "Specifies which branch to clone. " +
"If empty, will default to cloning the default branch or using the existing branch in the cloned repo on disk.",
Value: clibase.StringOf(&gitbranch),
Value: serpent.StringOf(&gitbranch),
},
{
Flag: "repo-dir",
Default: "dotfiles",
Env: "CODER_DOTFILES_REPO_DIR",
Description: "Specifies the directory for the dotfiles repository, relative to global config directory.",
Value: clibase.StringOf(&dotfilesRepoDir),
Value: serpent.StringOf(&dotfilesRepoDir),
},
cliui.SkipPromptOption(),
}
@@ -308,7 +308,7 @@ type ensureCorrectGitBranchParams struct {
gitBranch string
}
func ensureCorrectGitBranch(baseInv *clibase.Invocation, params ensureCorrectGitBranchParams) error {
func ensureCorrectGitBranch(baseInv *serpent.Invocation, params ensureCorrectGitBranchParams) error {
dotfileCmd := func(cmd string, args ...string) *exec.Cmd {
c := exec.CommandContext(baseInv.Context(), cmd, args...)
c.Dir = params.repoDir
+32 -26
View File
@@ -5,19 +5,18 @@ import (
"fmt"
"net/http"
"net/http/httptest"
"os"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
func (RootCmd) errorExample() *clibase.Cmd {
errorCmd := func(use string, err error) *clibase.Cmd {
return &clibase.Cmd{
func (RootCmd) errorExample() *serpent.Command {
errorCmd := func(use string, err error) *serpent.Command {
return &serpent.Command{
Use: use,
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
return err
},
}
@@ -45,22 +44,23 @@ func (RootCmd) errorExample() *clibase.Cmd {
apiError.(*codersdk.Error).Helper = "Have you tried turning it off and on again?"
//nolint:errorlint,forcetypeassert
apiErrorNoHelper := apiError.(*codersdk.Error)
cpy := *apiError.(*codersdk.Error)
apiErrorNoHelper := &cpy
apiErrorNoHelper.Helper = ""
// Some flags
var magicWord clibase.String
var magicWord serpent.String
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "example-error",
Short: "Shows what different error messages look like",
Long: "This command is pretty pointless, but without it testing errors is" +
"difficult to visually inspect. Error message formatting is inherently" +
"visual, so we need a way to quickly see what they look like.",
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
return inv.Command.HelpHandler(inv)
},
Children: []*clibase.Cmd{
Children: []*serpent.Command{
// Typical codersdk api error
errorCmd("api", apiError),
@@ -70,7 +70,7 @@ func (RootCmd) errorExample() *clibase.Cmd {
// A multi-error
{
Use: "multi-error",
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
return xerrors.Errorf("wrapped: %w", errors.Join(
xerrors.Errorf("first error: %w", errorWithStackTrace()),
xerrors.Errorf("second error: %w", errorWithStackTrace()),
@@ -81,39 +81,45 @@ func (RootCmd) errorExample() *clibase.Cmd {
{
Use: "multi-multi-error",
Short: "This is a multi error inside a multi error",
Handler: func(inv *clibase.Invocation) error {
// Closing the stdin file descriptor will cause the next close
// to fail. This is joined to the returned Command error.
if f, ok := inv.Stdin.(*os.File); ok {
_ = f.Close()
}
Handler: func(inv *serpent.Invocation) error {
return errors.Join(
xerrors.Errorf("first error: %w", errorWithStackTrace()),
xerrors.Errorf("second error: %w", errorWithStackTrace()),
xerrors.Errorf("parent error: %w", errorWithStackTrace()),
errors.Join(
xerrors.Errorf("child first error: %w", errorWithStackTrace()),
xerrors.Errorf("child second error: %w", errorWithStackTrace()),
),
)
},
},
{
Use: "validation",
Options: clibase.OptionSet{
clibase.Option{
Options: serpent.OptionSet{
serpent.Option{
Name: "magic-word",
Description: "Take a good guess.",
Required: true,
Flag: "magic-word",
Default: "",
Value: clibase.Validate(&magicWord, func(value *clibase.String) error {
Value: serpent.Validate(&magicWord, func(value *serpent.String) error {
return xerrors.Errorf("magic word is incorrect")
}),
},
},
Handler: func(i *clibase.Invocation) error {
Handler: func(i *serpent.Invocation) error {
_, _ = fmt.Fprint(i.Stdout, "Try setting the --magic-word flag\n")
return nil
},
},
{
Use: "arg-required <required>",
Middleware: serpent.Chain(
serpent.RequireNArgs(1),
),
Handler: func(i *serpent.Invocation) error {
_, _ = fmt.Fprint(i.Stdout, "Try running this without an argument\n")
return nil
},
},
},
}
+93
View File
@@ -0,0 +1,93 @@
package cli_test
import (
"bytes"
"fmt"
"strings"
"testing"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli"
"github.com/coder/coder/v2/cli/clitest"
"github.com/coder/serpent"
)
type commandErrorCase struct {
Name string
Cmd []string
}
// TestErrorExamples will test the help output of the
// coder exp example-error using golden files.
func TestErrorExamples(t *testing.T) {
t.Parallel()
coderRootCmd := getRoot(t)
var exampleErrorRootCmd *serpent.Command
coderRootCmd.Walk(func(command *serpent.Command) {
if command.Name() == "example-error" {
// cannot abort early, but list is small
exampleErrorRootCmd = command
}
})
require.NotNil(t, exampleErrorRootCmd, "example-error command not found")
var cases []commandErrorCase
ExtractCommandPathsLoop:
for _, cp := range extractCommandPaths(nil, exampleErrorRootCmd.Children) {
cmd := append([]string{"exp", "example-error"}, cp...)
name := fmt.Sprintf("coder %s", strings.Join(cmd, " "))
for _, tt := range cases {
if tt.Name == name {
continue ExtractCommandPathsLoop
}
}
cases = append(cases, commandErrorCase{Name: name, Cmd: cmd})
}
for _, tt := range cases {
tt := tt
t.Run(tt.Name, func(t *testing.T) {
t.Parallel()
var outBuf bytes.Buffer
coderRootCmd := getRoot(t)
inv, _ := clitest.NewWithCommand(t, coderRootCmd, tt.Cmd...)
inv.Stderr = &outBuf
inv.Stdout = &outBuf
err := inv.Run()
errFormatter := cli.NewPrettyErrorFormatter(&outBuf, false)
errFormatter.Format(err)
clitest.TestGoldenFile(t, tt.Name, outBuf.Bytes(), nil)
})
}
}
func extractCommandPaths(cmdPath []string, cmds []*serpent.Command) [][]string {
var cmdPaths [][]string
for _, c := range cmds {
cmdPath := append(cmdPath, c.Name())
cmdPaths = append(cmdPaths, cmdPath)
cmdPaths = append(cmdPaths, extractCommandPaths(cmdPath, c.Children)...)
}
return cmdPaths
}
// Must return a fresh instance of cmds each time.
func getRoot(t *testing.T) *serpent.Command {
t.Helper()
var root cli.RootCmd
rootCmd, err := root.Command(root.AGPL())
require.NoError(t, err)
return rootCmd
}
+5 -5
View File
@@ -1,16 +1,16 @@
package cli
import "github.com/coder/coder/v2/cli/clibase"
import "github.com/coder/serpent"
func (r *RootCmd) expCmd() *clibase.Cmd {
cmd := &clibase.Cmd{
func (r *RootCmd) expCmd() *serpent.Command {
cmd := &serpent.Command{
Use: "exp",
Short: "Internal commands for testing and experimentation. These are prone to breaking changes with no notice.",
Handler: func(i *clibase.Invocation) error {
Handler: func(i *serpent.Invocation) error {
return i.Command.HelpHandler(i)
},
Hidden: true,
Children: []*clibase.Cmd{
Children: []*serpent.Command{
r.scaletestCmd(),
r.errorExample(),
},
+92 -77
View File
@@ -27,11 +27,11 @@ import (
"cdr.dev/slog"
"cdr.dev/slog/sloggers/sloghuman"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/coderd/httpapi"
"github.com/coder/coder/v2/coderd/tracing"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
"github.com/coder/coder/v2/cryptorand"
"github.com/coder/coder/v2/scaletest/agentconn"
"github.com/coder/coder/v2/scaletest/createworkspaces"
@@ -40,18 +40,19 @@ import (
"github.com/coder/coder/v2/scaletest/reconnectingpty"
"github.com/coder/coder/v2/scaletest/workspacebuild"
"github.com/coder/coder/v2/scaletest/workspacetraffic"
"github.com/coder/serpent"
)
const scaletestTracerName = "coder_scaletest"
func (r *RootCmd) scaletestCmd() *clibase.Cmd {
cmd := &clibase.Cmd{
func (r *RootCmd) scaletestCmd() *serpent.Command {
cmd := &serpent.Command{
Use: "scaletest",
Short: "Run a scale test against the Coder API",
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
return inv.Command.HelpHandler(inv)
},
Children: []*clibase.Cmd{
Children: []*serpent.Command{
r.scaletestCleanup(),
r.scaletestDashboard(),
r.scaletestCreateWorkspaces(),
@@ -69,32 +70,32 @@ type scaletestTracingFlags struct {
tracePropagate bool
}
func (s *scaletestTracingFlags) attach(opts *clibase.OptionSet) {
func (s *scaletestTracingFlags) attach(opts *serpent.OptionSet) {
*opts = append(
*opts,
clibase.Option{
serpent.Option{
Flag: "trace",
Env: "CODER_SCALETEST_TRACE",
Description: "Whether application tracing data is collected. It exports to a backend configured by environment variables. See: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md.",
Value: clibase.BoolOf(&s.traceEnable),
Value: serpent.BoolOf(&s.traceEnable),
},
clibase.Option{
serpent.Option{
Flag: "trace-coder",
Env: "CODER_SCALETEST_TRACE_CODER",
Description: "Whether opentelemetry traces are sent to Coder. We recommend keeping this disabled unless we advise you to enable it.",
Value: clibase.BoolOf(&s.traceCoder),
Value: serpent.BoolOf(&s.traceCoder),
},
clibase.Option{
serpent.Option{
Flag: "trace-honeycomb-api-key",
Env: "CODER_SCALETEST_TRACE_HONEYCOMB_API_KEY",
Description: "Enables trace exporting to Honeycomb.io using the provided API key.",
Value: clibase.StringOf(&s.traceHoneycombAPIKey),
Value: serpent.StringOf(&s.traceHoneycombAPIKey),
},
clibase.Option{
serpent.Option{
Flag: "trace-propagate",
Env: "CODER_SCALETEST_TRACE_PROPAGATE",
Description: "Enables trace propagation to the Coder backend, which will be used to correlate server-side spans with client-side spans. Only enable this if the server is configured with the exact same tracing configuration as the client.",
Value: clibase.BoolOf(&s.tracePropagate),
Value: serpent.BoolOf(&s.tracePropagate),
},
)
}
@@ -137,7 +138,7 @@ type scaletestStrategyFlags struct {
timeoutPerJob time.Duration
}
func (s *scaletestStrategyFlags) attach(opts *clibase.OptionSet) {
func (s *scaletestStrategyFlags) attach(opts *serpent.OptionSet) {
concurrencyLong, concurrencyEnv, concurrencyDescription := "concurrency", "CODER_SCALETEST_CONCURRENCY", "Number of concurrent jobs to run. 0 means unlimited."
timeoutLong, timeoutEnv, timeoutDescription := "timeout", "CODER_SCALETEST_TIMEOUT", "Timeout for the entire test run. 0 means unlimited."
jobTimeoutLong, jobTimeoutEnv, jobTimeoutDescription := "job-timeout", "CODER_SCALETEST_JOB_TIMEOUT", "Timeout per job. Jobs may take longer to complete under higher concurrency limits."
@@ -149,26 +150,26 @@ func (s *scaletestStrategyFlags) attach(opts *clibase.OptionSet) {
*opts = append(
*opts,
clibase.Option{
serpent.Option{
Flag: concurrencyLong,
Env: concurrencyEnv,
Description: concurrencyDescription,
Default: "1",
Value: clibase.Int64Of(&s.concurrency),
Value: serpent.Int64Of(&s.concurrency),
},
clibase.Option{
serpent.Option{
Flag: timeoutLong,
Env: timeoutEnv,
Description: timeoutDescription,
Default: "30m",
Value: clibase.DurationOf(&s.timeout),
Value: serpent.DurationOf(&s.timeout),
},
clibase.Option{
serpent.Option{
Flag: jobTimeoutLong,
Env: jobTimeoutEnv,
Description: jobTimeoutDescription,
Default: "5m",
Value: clibase.DurationOf(&s.timeoutPerJob),
Value: serpent.DurationOf(&s.timeoutPerJob),
},
)
}
@@ -268,13 +269,13 @@ type scaletestOutputFlags struct {
outputSpecs []string
}
func (s *scaletestOutputFlags) attach(opts *clibase.OptionSet) {
*opts = append(*opts, clibase.Option{
func (s *scaletestOutputFlags) attach(opts *serpent.OptionSet) {
*opts = append(*opts, serpent.Option{
Flag: "output",
Env: "CODER_SCALETEST_OUTPUTS",
Description: `Output format specs in the format "<format>[:<path>]". Not specifying a path will default to stdout. Available formats: text, json.`,
Default: "text",
Value: clibase.StringArrayOf(&s.outputSpecs),
Value: serpent.StringArrayOf(&s.outputSpecs),
})
}
@@ -331,21 +332,21 @@ type scaletestPrometheusFlags struct {
Wait time.Duration
}
func (s *scaletestPrometheusFlags) attach(opts *clibase.OptionSet) {
func (s *scaletestPrometheusFlags) attach(opts *serpent.OptionSet) {
*opts = append(*opts,
clibase.Option{
serpent.Option{
Flag: "scaletest-prometheus-address",
Env: "CODER_SCALETEST_PROMETHEUS_ADDRESS",
Default: "0.0.0.0:21112",
Description: "Address on which to expose scaletest Prometheus metrics.",
Value: clibase.StringOf(&s.Address),
Value: serpent.StringOf(&s.Address),
},
clibase.Option{
serpent.Option{
Flag: "scaletest-prometheus-wait",
Env: "CODER_SCALETEST_PROMETHEUS_WAIT",
Default: "15s",
Description: "How long to wait before exiting in order to allow Prometheus metrics to be scraped.",
Value: clibase.DurationOf(&s.Wait),
Value: serpent.DurationOf(&s.Wait),
},
)
}
@@ -398,20 +399,20 @@ func (r *userCleanupRunner) Run(ctx context.Context, _ string, _ io.Writer) erro
return nil
}
func (r *RootCmd) scaletestCleanup() *clibase.Cmd {
func (r *RootCmd) scaletestCleanup() *serpent.Command {
var template string
cleanupStrategy := &scaletestStrategyFlags{cleanup: true}
client := new(codersdk.Client)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "cleanup",
Short: "Cleanup scaletest workspaces, then cleanup scaletest users.",
Long: "The strategy flags will apply to each stage of the cleanup process.",
Middleware: clibase.Chain(
Middleware: serpent.Chain(
r.InitClient(client),
),
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
ctx := inv.Context()
me, err := requireAdmin(ctx, client)
@@ -508,12 +509,12 @@ func (r *RootCmd) scaletestCleanup() *clibase.Cmd {
},
}
cmd.Options = clibase.OptionSet{
cmd.Options = serpent.OptionSet{
{
Flag: "template",
Env: "CODER_SCALETEST_CLEANUP_TEMPLATE",
Description: "Name or ID of the template. Only delete workspaces created from the given template.",
Value: clibase.StringOf(&template),
Value: serpent.StringOf(&template),
},
}
@@ -521,9 +522,10 @@ func (r *RootCmd) scaletestCleanup() *clibase.Cmd {
return cmd
}
func (r *RootCmd) scaletestCreateWorkspaces() *clibase.Cmd {
func (r *RootCmd) scaletestCreateWorkspaces() *serpent.Command {
var (
count int64
retry int64
template string
noCleanup bool
@@ -557,12 +559,12 @@ func (r *RootCmd) scaletestCreateWorkspaces() *clibase.Cmd {
client := new(codersdk.Client)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "create-workspaces",
Short: "Creates many users, then creates a workspace for each user and waits for them finish building and fully come online. Optionally runs a command inside each workspace, and connects to the workspace over WireGuard.",
Long: `It is recommended that all rate limits are disabled on the server before running this scaletest. This test generates many login events which will be rate limited against the (most likely single) IP.`,
Middleware: r.InitClient(client),
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
ctx := inv.Context()
me, err := requireAdmin(ctx, client)
@@ -644,6 +646,7 @@ func (r *RootCmd) scaletestCreateWorkspaces() *clibase.Cmd {
RichParameterValues: richParameters,
},
NoWaitForAgents: noWaitForAgents,
Retry: int(retry),
},
NoCleanup: noCleanup,
}
@@ -665,7 +668,7 @@ func (r *RootCmd) scaletestCreateWorkspaces() *clibase.Cmd {
if runCommand != "" {
config.ReconnectingPTY = &reconnectingpty.Config{
// AgentID is set by the test automatically.
Init: codersdk.WorkspaceAgentReconnectingPTYInit{
Init: workspacesdk.AgentReconnectingPTYInit{
ID: uuid.Nil,
Height: 24,
Width: 80,
@@ -744,91 +747,98 @@ func (r *RootCmd) scaletestCreateWorkspaces() *clibase.Cmd {
},
}
cmd.Options = clibase.OptionSet{
cmd.Options = serpent.OptionSet{
{
Flag: "count",
FlagShorthand: "c",
Env: "CODER_SCALETEST_COUNT",
Default: "1",
Description: "Required: Number of workspaces to create.",
Value: clibase.Int64Of(&count),
Value: serpent.Int64Of(&count),
},
{
Flag: "retry",
Env: "CODER_SCALETEST_RETRY",
Default: "0",
Description: "Number of tries to create and bring up the workspace.",
Value: serpent.Int64Of(&retry),
},
{
Flag: "template",
FlagShorthand: "t",
Env: "CODER_SCALETEST_TEMPLATE",
Description: "Required: Name or ID of the template to use for workspaces.",
Value: clibase.StringOf(&template),
Value: serpent.StringOf(&template),
},
{
Flag: "no-cleanup",
Env: "CODER_SCALETEST_NO_CLEANUP",
Description: "Do not clean up resources after the test completes. You can cleanup manually using coder scaletest cleanup.",
Value: clibase.BoolOf(&noCleanup),
Value: serpent.BoolOf(&noCleanup),
},
{
Flag: "no-wait-for-agents",
Env: "CODER_SCALETEST_NO_WAIT_FOR_AGENTS",
Description: `Do not wait for agents to start before marking the test as succeeded. This can be useful if you are running the test against a template that does not start the agent quickly.`,
Value: clibase.BoolOf(&noWaitForAgents),
Value: serpent.BoolOf(&noWaitForAgents),
},
{
Flag: "run-command",
Env: "CODER_SCALETEST_RUN_COMMAND",
Description: "Command to run inside each workspace using reconnecting-pty (i.e. web terminal protocol). " + "If not specified, no command will be run.",
Value: clibase.StringOf(&runCommand),
Value: serpent.StringOf(&runCommand),
},
{
Flag: "run-timeout",
Env: "CODER_SCALETEST_RUN_TIMEOUT",
Default: "5s",
Description: "Timeout for the command to complete.",
Value: clibase.DurationOf(&runTimeout),
Value: serpent.DurationOf(&runTimeout),
},
{
Flag: "run-expect-timeout",
Env: "CODER_SCALETEST_RUN_EXPECT_TIMEOUT",
Description: "Expect the command to timeout." + " If the command does not finish within the given --run-timeout, it will be marked as succeeded." + " If the command finishes before the timeout, it will be marked as failed.",
Value: clibase.BoolOf(&runExpectTimeout),
Value: serpent.BoolOf(&runExpectTimeout),
},
{
Flag: "run-expect-output",
Env: "CODER_SCALETEST_RUN_EXPECT_OUTPUT",
Description: "Expect the command to output the given string (on a single line). " + "If the command does not output the given string, it will be marked as failed.",
Value: clibase.StringOf(&runExpectOutput),
Value: serpent.StringOf(&runExpectOutput),
},
{
Flag: "run-log-output",
Env: "CODER_SCALETEST_RUN_LOG_OUTPUT",
Description: "Log the output of the command to the test logs. " + "This should be left off unless you expect small amounts of output. " + "Large amounts of output will cause high memory usage.",
Value: clibase.BoolOf(&runLogOutput),
Value: serpent.BoolOf(&runLogOutput),
},
{
Flag: "connect-url",
Env: "CODER_SCALETEST_CONNECT_URL",
Description: "URL to connect to inside the the workspace over WireGuard. " + "If not specified, no connections will be made over WireGuard.",
Value: clibase.StringOf(&connectURL),
Value: serpent.StringOf(&connectURL),
},
{
Flag: "connect-mode",
Env: "CODER_SCALETEST_CONNECT_MODE",
Default: "derp",
Description: "Mode to use for connecting to the workspace.",
Value: clibase.EnumOf(&connectMode, "derp", "direct"),
Value: serpent.EnumOf(&connectMode, "derp", "direct"),
},
{
Flag: "connect-hold",
Env: "CODER_SCALETEST_CONNECT_HOLD",
Default: "30s",
Description: "How long to hold the WireGuard connection open for.",
Value: clibase.DurationOf(&connectHold),
Value: serpent.DurationOf(&connectHold),
},
{
Flag: "connect-interval",
Env: "CODER_SCALETEST_CONNECT_INTERVAL",
Default: "1s",
Value: clibase.DurationOf(&connectInterval),
Value: serpent.DurationOf(&connectInterval),
Description: "How long to wait between making requests to the --connect-url once the connection is established.",
},
{
@@ -836,14 +846,14 @@ func (r *RootCmd) scaletestCreateWorkspaces() *clibase.Cmd {
Env: "CODER_SCALETEST_CONNECT_TIMEOUT",
Default: "5s",
Description: "Timeout for each request to the --connect-url.",
Value: clibase.DurationOf(&connectTimeout),
Value: serpent.DurationOf(&connectTimeout),
},
{
Flag: "use-host-login",
Env: "CODER_SCALETEST_USE_HOST_LOGIN",
Default: "false",
Description: "Use the user logged in on the host machine, instead of creating users.",
Value: clibase.BoolOf(&useHostUser),
Value: serpent.BoolOf(&useHostUser),
},
}
@@ -855,7 +865,7 @@ func (r *RootCmd) scaletestCreateWorkspaces() *clibase.Cmd {
return cmd
}
func (r *RootCmd) scaletestWorkspaceTraffic() *clibase.Cmd {
func (r *RootCmd) scaletestWorkspaceTraffic() *serpent.Command {
var (
tickInterval time.Duration
bytesPerTick int64
@@ -872,16 +882,16 @@ func (r *RootCmd) scaletestWorkspaceTraffic() *clibase.Cmd {
prometheusFlags = &scaletestPrometheusFlags{}
)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "workspace-traffic",
Short: "Generate traffic to scaletest workspaces through coderd",
Middleware: clibase.Chain(
Middleware: serpent.Chain(
r.InitClient(client),
),
Handler: func(inv *clibase.Invocation) (err error) {
Handler: func(inv *serpent.Invocation) (err error) {
ctx := inv.Context()
notifyCtx, stop := signal.NotifyContext(ctx, InterruptSignals...) // Checked later.
notifyCtx, stop := signal.NotifyContext(ctx, StopSignals...) // Checked later.
defer stop()
ctx = notifyCtx
@@ -1047,47 +1057,47 @@ func (r *RootCmd) scaletestWorkspaceTraffic() *clibase.Cmd {
},
}
cmd.Options = []clibase.Option{
cmd.Options = []serpent.Option{
{
Flag: "template",
FlagShorthand: "t",
Env: "CODER_SCALETEST_TEMPLATE",
Description: "Name or ID of the template. Traffic generation will be limited to workspaces created from this template.",
Value: clibase.StringOf(&template),
Value: serpent.StringOf(&template),
},
{
Flag: "target-workspaces",
Env: "CODER_SCALETEST_TARGET_WORKSPACES",
Description: "Target a specific range of workspaces in the format [START]:[END] (exclusive). Example: 0:10 will target the 10 first alphabetically sorted workspaces (0-9).",
Value: clibase.StringOf(&targetWorkspaces),
Value: serpent.StringOf(&targetWorkspaces),
},
{
Flag: "bytes-per-tick",
Env: "CODER_SCALETEST_WORKSPACE_TRAFFIC_BYTES_PER_TICK",
Default: "1024",
Description: "How much traffic to generate per tick.",
Value: clibase.Int64Of(&bytesPerTick),
Value: serpent.Int64Of(&bytesPerTick),
},
{
Flag: "tick-interval",
Env: "CODER_SCALETEST_WORKSPACE_TRAFFIC_TICK_INTERVAL",
Default: "100ms",
Description: "How often to send traffic.",
Value: clibase.DurationOf(&tickInterval),
Value: serpent.DurationOf(&tickInterval),
},
{
Flag: "ssh",
Env: "CODER_SCALETEST_WORKSPACE_TRAFFIC_SSH",
Default: "",
Description: "Send traffic over SSH, cannot be used with --app.",
Value: clibase.BoolOf(&ssh),
Value: serpent.BoolOf(&ssh),
},
{
Flag: "app",
Env: "CODER_SCALETEST_WORKSPACE_TRAFFIC_APP",
Default: "",
Description: "Send WebSocket traffic to a workspace app (proxied via coderd), cannot be used with --ssh.",
Value: clibase.StringOf(&app),
Value: serpent.StringOf(&app),
},
}
@@ -1100,7 +1110,7 @@ func (r *RootCmd) scaletestWorkspaceTraffic() *clibase.Cmd {
return cmd
}
func (r *RootCmd) scaletestDashboard() *clibase.Cmd {
func (r *RootCmd) scaletestDashboard() *serpent.Command {
var (
interval time.Duration
jitter time.Duration
@@ -1116,13 +1126,13 @@ func (r *RootCmd) scaletestDashboard() *clibase.Cmd {
prometheusFlags = &scaletestPrometheusFlags{}
)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "dashboard",
Short: "Generate traffic to the HTTP API to simulate use of the dashboard.",
Middleware: clibase.Chain(
Middleware: serpent.Chain(
r.InitClient(client),
),
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
if !(interval > 0) {
return xerrors.Errorf("--interval must be greater than zero")
}
@@ -1205,6 +1215,11 @@ func (r *RootCmd) scaletestDashboard() *clibase.Cmd {
// This could be useful for debugging, but it will blow up the disk.
if r.verbose {
config.Screenshot = dashboard.Screenshot
} else {
// Disable screenshots otherwise.
config.Screenshot = func(context.Context, string) (string, error) {
return "/dev/null", nil
}
}
//nolint:gocritic
logger.Info(ctx, "runner config", slog.F("interval", interval), slog.F("jitter", jitter), slog.F("headless", headless), slog.F("trace", tracingEnabled))
@@ -1247,40 +1262,40 @@ func (r *RootCmd) scaletestDashboard() *clibase.Cmd {
},
}
cmd.Options = []clibase.Option{
cmd.Options = []serpent.Option{
{
Flag: "target-users",
Env: "CODER_SCALETEST_DASHBOARD_TARGET_USERS",
Description: "Target a specific range of users in the format [START]:[END] (exclusive). Example: 0:10 will target the 10 first alphabetically sorted users (0-9).",
Value: clibase.StringOf(&targetUsers),
Value: serpent.StringOf(&targetUsers),
},
{
Flag: "interval",
Env: "CODER_SCALETEST_DASHBOARD_INTERVAL",
Default: "10s",
Description: "Interval between actions.",
Value: clibase.DurationOf(&interval),
Value: serpent.DurationOf(&interval),
},
{
Flag: "jitter",
Env: "CODER_SCALETEST_DASHBOARD_JITTER",
Default: "5s",
Description: "Jitter between actions.",
Value: clibase.DurationOf(&jitter),
Value: serpent.DurationOf(&jitter),
},
{
Flag: "headless",
Env: "CODER_SCALETEST_DASHBOARD_HEADLESS",
Default: "true",
Description: "Controls headless mode. Setting to false is useful for debugging.",
Value: clibase.BoolOf(&headless),
Value: serpent.BoolOf(&headless),
},
{
Flag: "rand-seed",
Env: "CODER_SCALETEST_DASHBOARD_RAND_SEED",
Default: "0",
Description: "Seed for the random number generator.",
Value: clibase.Int64Of(&randSeed),
Value: serpent.Int64Of(&randSeed),
},
}
+4 -4
View File
@@ -2,13 +2,13 @@
package cli
import "github.com/coder/coder/v2/cli/clibase"
import "github.com/coder/serpent"
func (r *RootCmd) scaletestCmd() *clibase.Cmd {
cmd := &clibase.Cmd{
func (r *RootCmd) scaletestCmd() *serpent.Command {
cmd := &serpent.Command{
Use: "scaletest",
Short: "Run a scale test against the Coder API",
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
SlimUnsupported(inv.Stderr, "exp scaletest")
return nil
},
+21 -11
View File
@@ -2,33 +2,35 @@ package cli
import (
"encoding/json"
"fmt"
"golang.org/x/xerrors"
"github.com/tidwall/gjson"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/pretty"
"github.com/coder/serpent"
)
func (r *RootCmd) externalAuth() *clibase.Cmd {
return &clibase.Cmd{
func (r *RootCmd) externalAuth() *serpent.Command {
return &serpent.Command{
Use: "external-auth",
Short: "Manage external authentication",
Long: "Authenticate with external services inside of a workspace.",
Handler: func(i *clibase.Invocation) error {
Handler: func(i *serpent.Invocation) error {
return i.Command.HelpHandler(i)
},
Children: []*clibase.Cmd{
Children: []*serpent.Command{
r.externalAuthAccessToken(),
},
}
}
func (r *RootCmd) externalAuthAccessToken() *clibase.Cmd {
func (r *RootCmd) externalAuthAccessToken() *serpent.Command {
var extra string
return &clibase.Cmd{
return &serpent.Command{
Use: "access-token <provider>",
Short: "Print auth for an external provider",
Long: "Print an access-token for an external auth provider. " +
@@ -52,19 +54,27 @@ fi
Command: "coder external-auth access-token slack --extra \"authed_user.id\"",
},
),
Options: clibase.OptionSet{{
Middleware: serpent.Chain(
serpent.RequireNArgs(1),
),
Options: serpent.OptionSet{{
Name: "Extra",
Flag: "extra",
Description: "Extract a field from the \"extra\" properties of the OAuth token.",
Value: clibase.StringOf(&extra),
Value: serpent.StringOf(&extra),
}},
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
ctx := inv.Context()
ctx, stop := inv.SignalNotifyContext(ctx, InterruptSignals...)
ctx, stop := inv.SignalNotifyContext(ctx, StopSignals...)
defer stop()
if r.agentToken == "" {
_, _ = fmt.Fprint(inv.Stderr, pretty.Sprintf(headLineStyle(), "No agent token found, this command must be run from inside a running workspace.\n"))
return xerrors.Errorf("agent token not found")
}
client, err := r.createAgentClient()
if err != nil {
return xerrors.Errorf("create agent client: %w", err)
+16 -3
View File
@@ -24,7 +24,7 @@ func TestExternalAuth(t *testing.T) {
}))
t.Cleanup(srv.Close)
url := srv.URL
inv, _ := clitest.New(t, "--agent-url", url, "external-auth", "access-token", "github")
inv, _ := clitest.New(t, "--agent-url", url, "--agent-token", "foo", "external-auth", "access-token", "github")
pty := ptytest.New(t)
inv.Stdout = pty.Output()
waiter := clitest.StartWithWaiter(t, inv)
@@ -40,12 +40,25 @@ func TestExternalAuth(t *testing.T) {
}))
t.Cleanup(srv.Close)
url := srv.URL
inv, _ := clitest.New(t, "--agent-url", url, "external-auth", "access-token", "github")
inv, _ := clitest.New(t, "--agent-url", url, "--agent-token", "foo", "external-auth", "access-token", "github")
pty := ptytest.New(t)
inv.Stdout = pty.Output()
clitest.Start(t, inv)
pty.ExpectMatch("bananas")
})
t.Run("NoArgs", func(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
httpapi.Write(context.Background(), w, http.StatusOK, agentsdk.ExternalAuthResponse{
AccessToken: "bananas",
})
}))
t.Cleanup(srv.Close)
url := srv.URL
inv, _ := clitest.New(t, "--agent-url", url, "--agent-token", "foo", "external-auth", "access-token")
watier := clitest.StartWithWaiter(t, inv)
watier.RequireContains("wanted 1 args but got 0")
})
t.Run("SuccessWithExtra", func(t *testing.T) {
t.Parallel()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -58,7 +71,7 @@ func TestExternalAuth(t *testing.T) {
}))
t.Cleanup(srv.Close)
url := srv.URL
inv, _ := clitest.New(t, "--agent-url", url, "external-auth", "access-token", "github", "--extra", "hey")
inv, _ := clitest.New(t, "--agent-url", url, "--agent-token", "foo", "external-auth", "access-token", "github", "--extra", "hey")
pty := ptytest.New(t)
inv.Stdout = pty.Output()
clitest.Start(t, inv)
+64
View File
@@ -0,0 +1,64 @@
package cli
import (
"fmt"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
func (r *RootCmd) favorite() *serpent.Command {
client := new(codersdk.Client)
cmd := &serpent.Command{
Aliases: []string{"fav", "favou" + "rite"},
Annotations: workspaceCommand,
Use: "favorite <workspace>",
Short: "Add a workspace to your favorites",
Middleware: serpent.Chain(
serpent.RequireNArgs(1),
r.InitClient(client),
),
Handler: func(inv *serpent.Invocation) error {
ws, err := namedWorkspace(inv.Context(), client, inv.Args[0])
if err != nil {
return xerrors.Errorf("get workspace: %w", err)
}
if err := client.FavoriteWorkspace(inv.Context(), ws.ID); err != nil {
return xerrors.Errorf("favorite workspace: %w", err)
}
_, _ = fmt.Fprintf(inv.Stdout, "Workspace %q added to favorites.\n", ws.Name)
return nil
},
}
return cmd
}
func (r *RootCmd) unfavorite() *serpent.Command {
client := new(codersdk.Client)
cmd := &serpent.Command{
Aliases: []string{"unfav", "unfavou" + "rite"},
Annotations: workspaceCommand,
Use: "unfavorite <workspace>",
Short: "Remove a workspace from your favorites",
Middleware: serpent.Chain(
serpent.RequireNArgs(1),
r.InitClient(client),
),
Handler: func(inv *serpent.Invocation) error {
ws, err := namedWorkspace(inv.Context(), client, inv.Args[0])
if err != nil {
return xerrors.Errorf("get workspace: %w", err)
}
if err := client.UnfavoriteWorkspace(inv.Context(), ws.ID); err != nil {
return xerrors.Errorf("unfavorite workspace: %w", err)
}
_, _ = fmt.Fprintf(inv.Stdout, "Workspace %q removed from favorites.\n", ws.Name)
return nil
},
}
return cmd
}
+45
View File
@@ -0,0 +1,45 @@
package cli_test
import (
"bytes"
"testing"
"github.com/coder/coder/v2/cli/clitest"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/dbfake"
"github.com/stretchr/testify/require"
)
func TestFavoriteUnfavorite(t *testing.T) {
t.Parallel()
var (
client, db = coderdtest.NewWithDatabase(t, nil)
owner = coderdtest.CreateFirstUser(t, client)
memberClient, member = coderdtest.CreateAnotherUser(t, client, owner.OrganizationID)
ws = dbfake.WorkspaceBuild(t, db, database.Workspace{OwnerID: member.ID, OrganizationID: owner.OrganizationID}).Do()
)
inv, root := clitest.New(t, "favorite", ws.Workspace.Name)
clitest.SetupConfig(t, memberClient, root)
var buf bytes.Buffer
inv.Stdout = &buf
err := inv.Run()
require.NoError(t, err)
updated := coderdtest.MustWorkspace(t, memberClient, ws.Workspace.ID)
require.True(t, updated.Favorite)
buf.Reset()
inv, root = clitest.New(t, "unfavorite", ws.Workspace.Name)
clitest.SetupConfig(t, memberClient, root)
inv.Stdout = &buf
err = inv.Run()
require.NoError(t, err)
updated = coderdtest.MustWorkspace(t, memberClient, ws.Workspace.ID)
require.False(t, updated.Favorite)
}
+5 -5
View File
@@ -8,24 +8,24 @@ import (
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/cli/gitauth"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/retry"
"github.com/coder/serpent"
)
// gitAskpass is used by the Coder agent to automatically authenticate
// with Git providers based on a hostname.
func (r *RootCmd) gitAskpass() *clibase.Cmd {
return &clibase.Cmd{
func (r *RootCmd) gitAskpass() *serpent.Command {
return &serpent.Command{
Use: "gitaskpass",
Hidden: true,
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
ctx := inv.Context()
ctx, stop := inv.SignalNotifyContext(ctx, InterruptSignals...)
ctx, stop := inv.SignalNotifyContext(ctx, StopSignals...)
defer stop()
user, host, err := gitauth.ParseAskpass(inv.Args[0])

Some files were not shown because too many files have changed in this diff Show More