Compare commits

...

1106 Commits

Author SHA1 Message Date
Kyle Carberry fefb4e655d fix: write server config to telemetry (#13590)
* fix: add external auth configs to telemetry

* Refactor telemetry to send the entire config

* gen

* Fix linting

(cherry picked from commit 3a1fa04590)
2024-06-21 19:03:35 +00:00
Kyle Carberry 62fdea0aa0 fix: display trial errors in the dashboard (#13601)
* fix: display trial errors in the dashboard

The error was essentially being ignored before!

* Remove day mention in product of trial

* fmt

(cherry picked from commit 7049d7a881)
2024-06-21 19:03:15 +00:00
Kyle Carberry 350c71b221 feat: add cross-origin reporting for telemetry in the dashboard (#13612)
* feat: add cross-origin reporting for telemetry in the dashboard

* Respect the telemetry flag

* Fix embedded metadata

* Fix compilation error

* Fix linting

(cherry picked from commit 0793a4b35b)
2024-06-21 19:03:10 +00:00
Kyle Carberry 203a182934 fix: track login page correctly (#13618)
(cherry picked from commit 495eea452f)
2024-06-21 19:03:04 +00:00
Kyle Carberry bfb4a8df11 fix: remove connected button (#13625)
It didn't make a lot of sense in current form. It will when we improve autostop.

(cherry picked from commit 3ef12ac284)
2024-06-21 19:02:58 +00:00
Mathias Fredriksson 9424fce5a5 chore(scripts): fix release promote stable to set latest tag (#13471)
(cherry picked from commit 9a757f8e74)
2024-06-21 19:02:30 +00:00
Mathias Fredriksson 9151f53591 chore(scripts): fix dry run for autoversion in release.sh (#13470)
(cherry picked from commit 3b7f9534fb)
2024-06-21 19:02:24 +00:00
Mathias Fredriksson 94c3dd8392 chore(scripts): fix unbound variable in tag_version.sh (#13428)
(cherry picked from commit a51076a4cd)
2024-06-21 19:02:18 +00:00
Mathias Fredriksson b8b2ed07b6 chore(scripts): fix expression interpreted as exit code on some Bash versions (#13417)
(cherry picked from commit 4758952ebc)
2024-06-21 19:02:12 +00:00
Mathias Fredriksson 360ff1c137 chore(scripts): add safety check for difference between dry run release notes (#13398)
(cherry picked from commit 9eb797eb5a)
2024-06-21 19:02:06 +00:00
Mathias Fredriksson 19b1390a3a chore(scripts): push version bump pr branch in release script (#13397)
(cherry picked from commit 9ae825ebae)
2024-06-21 19:01:57 +00:00
Mathias Fredriksson 4281e43090 chore(scripts): add custom gh auth to release script (#13396)
(cherry picked from commit 5fb231774c)
2024-06-21 19:01:38 +00:00
Mathias Fredriksson 66de7dd387 chore(scripts): handle renamed cherry-pick commits in release script (#13395)
(cherry picked from commit 374f0a0fd1)
2024-06-21 19:01:08 +00:00
Mathias Fredriksson c3221cec81 chore(scripts): fix stable release promote script (#13204)
(cherry picked from commit f66d0445da)
2024-06-21 19:00:56 +00:00
Colin Adler d53c94b32a fix: prevent stdlib logging from messing up ssh (#13161)
Fixes https://github.com/coder/coder/issues/13144

(cherry picked from commit 13dd526f11)
2024-05-22 18:12:23 +00:00
Mathias Fredriksson d31e07d605 chore(scripts): fix a few release script changelog issues (#13200)
(cherry picked from commit 0998cedb5c)
2024-05-22 18:10:18 +00:00
Kyle Carberry 9c5fc3bbbb fix: properly detect agent resouces in terraform (#13343)
Terraform changed the default output of the `terraform graph` command.
You must put `-type=plan` to keep the prior behavior.

(cherry picked from commit 3364abecdd)

Co-authored-by: Colin Adler <colin1adler@gmail.com>
2024-05-22 18:04:28 +00:00
Colin Adler d56514492b security: update git -> 2.43.4 and terraform -> 1.7.5 (#13299)
This fixes an RCE in git and gets us one minor version closer to fixing
a critical Terraform vulnerability. In the next release we'll bump to
1.8.x.

(cherry picked from commit 80538c079d)
2024-05-16 19:18:05 +00:00
Mathias Fredriksson 8979bfe059 ci: disable make test-migrations in release.yaml (#13201) 2024-05-07 17:30:19 +00:00
Colin Adler ebacced232 fix(enterprise): mark nodes from unhealthy coordinators as lost (#13123)
Instead of removing the mappings of unhealthy coordinators entirely,
mark them as lost instead. This prevents peers from disappearing from
other peers if a coordinator misses a heartbeat.
2024-05-06 19:47:44 +00:00
Garrett Delfosse a3c23ed313 chore: add docs for sharing ports (#13136)
Co-authored-by: kirby <kirby@coder.com>
Co-authored-by: Stephen Kirby <me@skirby.dev>
2024-05-03 12:38:29 -04:00
Michael Smith 34a3bdc4ec fix: add more tests for metadata hook functionality (#13145) 2024-05-03 15:28:54 +00:00
Steven Masley 09f00c08df chore: shutdown provisioner should stop waiting on client (#13118)
* chore: shutdown provisioner should stop waiting on client
* chore: add unit test that replicates failed client conn
2024-05-03 10:15:17 -05:00
Steven Masley 94a3e3a563 chore: allow terraform & echo built-in provisioners (#13121)
* chore: allow terraform & echo built-in provisioners

Built-in provisioners serve all specified types. This allows running terraform, echo, or both in built in.
The cli flag to control the types is hidden by default, to be used primarily for testing purposes.
2024-05-03 10:14:26 -05:00
Michael Smith 7873c961e3 fix: ensure signing out cannot cause any runtime render errors (#13137)
* fix: remove some of the jank around our core App component

* refactor: scope navigation logic more aggressively

* refactor: add explicit return type to useAuthenticated

* refactor: clean up ProxyContext code

* wip: add code for consolidating the HTML metadata

* refactor: clean up hook logic

* refactor: rename useHtmlMetadata to useEmbeddedMetadata

* fix: correct names that weren't updated

* fix: update type-safety of useEmbeddedMetadata further

* wip: switch codebase to use metadata hook

* refactor: simplify design of metadata hook

* fix: update stray type mismatches

* fix: more type fixing

* fix: resolve illegal invocation error

* fix: get metadata issue resolved

* fix: update comments

* chore: add unit tests for MetadataManager

* fix: beef up tests

* fix: update typo in tests
2024-05-03 10:40:06 -04:00
Dean Sheather ed0ca76b0b chore: do network integration tests in isolated net ns (#13117) 2024-05-03 05:42:13 +00:00
Steven Masley 7779c0a1dc chore: enable playwright test extension in vscode (#13135)
* chore: enable playwright test extension in vscode

This enables using the vscode debugger in playwright tests
2024-05-02 23:14:24 +00:00
Garrett Delfosse 699e187d55 fix: remove mention of protocol lag (#13133) 2024-05-02 17:10:30 -04:00
Eric Paulsen 565b45deba docs: add island integration guide (#13113)
* docs: add island integration guide

* make: fmt

* F

omit F

* fix: naming and manifest

---------

Co-authored-by: Matt Vollmer <matthewjvollmer@outlook.com>
2024-05-02 15:12:34 -04:00
Garrett Delfosse c550d0641d feat: move shared ports out of experiment (#13120) 2024-05-02 14:11:33 -04:00
Steven Masley c2cb0e9fe2 chore: testIDP to be usable as primary auth (#13132)
Flags printed to console show as external or primary auth.
Usage assumes only 1 static oidc_member for now
2024-05-02 11:19:19 -05:00
Spike Curtis 3de737fdc8 fix: start packet capture immediately on speedtest (#13128)
I initially made this change when hacking wgengine to also capture wireguard packets going into the magicsock, so that we could capture the initial wireguard handshake. 

I don't think we should ship that additional capture logic, but... it seems generally useful to capture packets from the get go on speedtest, so that you can see disco and pings before the TCP speedtest session starts.
2024-05-02 19:44:32 +04:00
Kyle Carberry 93d8812284 chore: remove codecov (#13124)
* chore: remove codecov

It wasn't being used anymore.

* Update actions packages
2024-05-01 21:47:25 +00:00
Steven Masley 845407fe7a chore: cover deadline crossing autostart border on start (#13115)
When starting a workspace, if the deadline crosses an autostart boundary, the deadline is set to autostart + TTL. 
This copies the behavior in `ActivityBumpWorkspace`, but does not require activity.
2024-05-01 10:43:04 -05:00
Bruno Quaresma 71a03a8b1d fix(site): fix template schedule update overriding other settings (#13114) 2024-05-01 10:25:40 -03:00
Dean Sheather f2dd0a8e5d feat: try IPv6 when dialing IPv4 in workspaces (#13116) 2024-05-01 21:45:25 +10:00
Muhammad Atif Ali 3ff9cef498 chore(scripts): auto authenticate gh CLI in scripts on dogfood (#13107)
* chore: auto authenticate gh CLI in scripts

* fix shellcheck issues
2024-04-30 19:36:12 +03:00
Steven Masley 53f7e9e0a1 chore: dynamically determine gitlab external auth defaults (#13102)
* chore: dynamically determine gitlab external auth defaults

Static defaults work for github cloud, but not self hosted.
Self hosted setups will now have sane defaults if omitted.
2024-04-30 09:45:52 -05:00
Kyle Carberry 47993e3fcf chore: update tailscale to fix leaking dns lookup (#13109)
See failure in: https://github.com/coder/coder/actions/runs/8887860105/job/24403798734#step:5:376
2024-04-30 12:36:16 +00:00
Kyle Carberry d302570091 chore: remove GITHUB_TOKEN from dogfood env vars (#13106)
This was stale all the time!
2024-04-30 01:26:58 +00:00
Kyle Carberry 4e5960660e chore: fix dependency review action (#13105)
See https://github.com/actions/dependency-review-action/issues/757
2024-04-29 20:51:01 -04:00
Kyle Carberry fbb98b950a chore: centralize build info for site (#13104)
The build info passed to the frontend via HTML was incorrect.
2024-04-29 20:50:49 -04:00
Kyle Carberry 1bda8a0856 feat: add deployment_id to the ui and licenses (#13096)
* feat: expose `deployment_id` in the user dropdown

* feat: add license deployment_id verification

* Ignore wireguard.com from mlc config
2024-04-29 16:50:11 -04:00
Aaron Lehmann 0e3dc2a80f feat: influence parameter defaults through cli flag/env (#13039)
* feat: influence parameter defaults through cli flag/env

Add a --parameter-default flag / CODER_RICH_PARAMETER_DEFAULT
environment variable which overrides default values suggested for
parameters.

This allows scripts or middleware wrapping the CLI to substitute
defaults for parameter values beyond those defined at the template
level. For example, Git repository/branch parameters can be given
defaults based on the current checkout, or default parameter values can
be parsed out of files inside the repo.

* Rename defaults arg to defaultOverrides
2024-04-29 14:23:54 -04:00
Bruno Quaresma 053c56cc1a fix(site): fix template schedule options (#13084) 2024-04-29 14:14:24 -03:00
dependabot[bot] ed07921752 ci: bump crate-ci/typos from 1.20.9 to 1.20.10 in the github-actions group (#13090)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-29 16:57:53 +00:00
dependabot[bot] 4a83e84a23 chore: bump github.com/jmoiron/sqlx from 1.3.5 to 1.4.0 (#13095)
Bumps [github.com/jmoiron/sqlx](https://github.com/jmoiron/sqlx) from 1.3.5 to 1.4.0.
- [Release notes](https://github.com/jmoiron/sqlx/releases)
- [Commits](https://github.com/jmoiron/sqlx/compare/v1.3.5...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/jmoiron/sqlx
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-29 19:54:40 +03:00
dependabot[bot] f2a21c604b chore: bump github.com/moby/moby (#13093)
Bumps [github.com/moby/moby](https://github.com/moby/moby) from 26.0.1+incompatible to 26.1.0+incompatible.
- [Release notes](https://github.com/moby/moby/releases)
- [Commits](https://github.com/moby/moby/compare/v26.0.1...v26.1.0)

---
updated-dependencies:
- dependency-name: github.com/moby/moby
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-29 19:54:27 +03:00
dependabot[bot] 74b921cf81 chore: bump google.golang.org/api from 0.175.0 to 0.176.1 (#13092)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.175.0 to 0.176.1.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.175.0...v0.176.1)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-29 16:54:20 +00:00
Kyle Carberry 1b3185c047 chore: lower azure expires soon (#13097)
They haven't provisioned new certificates yet.
2024-04-29 12:34:18 -04:00
Jon Ayers 8269124ab7 feat: sign windows binaries (#13086) 2024-04-29 10:43:27 -05:00
Colin Adler 15157c1c40 chore: add network integration test suite scaffolding (#13072)
* chore: add network integration test suite scaffolding

* dean comments
2024-04-26 17:48:41 +00:00
Cian Johnston 73ba36c9d2 chore(docs): add note regarding Apr 26 scaletest (#13085) 2024-04-26 17:06:36 +01:00
Garrett Delfosse 8ba05a9052 feat: add switch http(s) button to error page (#12942) 2024-04-26 11:52:53 -04:00
Michael Brewer 848ea7e9f1 chore: correct name for github enterprise example (#13083)
Co-authored-by: Muhammad Atif Ali <me@matifali.dev>
2024-04-26 14:43:28 +00:00
Cian Johnston f1ef9fd673 chore(docs): add note regarding vcredist for embedded postgres (#13020) 2024-04-26 10:56:43 +01:00
Mathias Fredriksson d50a31ef62 chore(scripts): auto create autoversion PR from release script (#13074)
Ref #12465
2024-04-26 12:53:22 +03:00
Cian Johnston 365231b1e5 fix(cli): scaletest: ignore errors syncing output (#13076) 2024-04-26 09:18:33 +01:00
Kayla Washburn-Love 74f27719b8 feat: specify a custom "terms of service" link (#13068) 2024-04-25 16:36:51 -06:00
Stephen Kirby 341114a020 chore(docs): remove max_ttl docs (#13077)
* removed MAX_TTL docs, updated template-level scheduling controls

* fmt
2024-04-25 16:13:42 -05:00
Cian Johnston 99dda4a43a fix(agent): keep track of lastReportIndex between invocations of reportLifecycle() (#13075) 2024-04-25 16:54:51 +01:00
Mathias Fredriksson c24b562199 chore(scripts): fix release tagging sanity checks (#13073) 2024-04-25 12:26:37 +03:00
Mathias Fredriksson 46dced9cfe chore(scripts): add release autoversion to bump releases in docs (#13063)
This PR adds a command to bump versions in docs/markdown.

This is still standalone and needs to be wired up.

For now, I'm planning on putting this in `scripts/release.sh` (checkout main -> autoversion (this command) -> commit -> submit PR).

It would be pretty neat to make it a GH actions that's triggered on release though, something for the future.

Part of #12465
2024-04-25 12:11:55 +03:00
Mathias Fredriksson c933c75aa7 chore(scripts): add script to promote mainline to stable (#13054)
Fixes #12459

Example dry-run:

<img width="1229" alt="Screenshot 2024-04-23 at 21 16 55" src="https://github.com/coder/coder/assets/147409/7018d322-501b-41e2-bf47-af3fc39fb3d2">

Example dry-run for non-latest version:

<img width="1228" alt="Screenshot 2024-04-23 at 21 17 52" src="https://github.com/coder/coder/assets/147409/a05fcd44-560f-4e44-81b5-76c071c591b4">

**Note:** This PR does not yet update docs to reflect the promoted version. This will be part of #12465.
2024-04-24 22:59:22 +03:00
Mathias Fredriksson b82a782619 chore(scripts): implement mainline and stable release channels (#13048)
Fixes #12458
2024-04-24 19:43:11 +00:00
Frederik Dudzik a6af7a5e3d chore(README): add contributing section to readme (#13059) 2024-04-24 22:15:14 +03:00
Michael Smith 3f21cb8a2f fix: update API code to use Axios instances (#13029)
* fix: update API code to use Axios instance

* docs: fix typo

* fix: update all global axios imports to use Coder instance

* fix: remove needless import

* fix: update import order

* refactor: rename coderAxiosInstance to axiosInstance

* docs: update variable reference in FE contributing docs
2024-04-24 17:01:23 +00:00
Stephen Kirby dd27a8a634 updated helm install flags to match patches (#13064) 2024-04-24 10:26:36 -05:00
dependabot[bot] 39ccff97c1 chore: bump github.com/gohugoio/hugo from 0.125.2 to 0.125.3 (#13057)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-24 13:39:52 +00:00
Frederik Dudzik 8b6227d031 chore(docs): add k8s terraform link to k8s installation docs (#13040)
* add k8s terraform link to k8s installation docs

* Update kubernetes.md
2024-04-24 06:28:16 -04:00
Cian Johnston a518047f10 chore(coderd): provisionerdserver: downgrade heartbeat failure log to Warn instead of Error (#13061) 2024-04-24 09:36:36 +01:00
Pavel Aseev 4682355eed chore: deprecate gauge metrics with _total suffix (#12744) (#12976)
* chore: deprecate gauge metrics with _total suffix (#12744)

Deprecated metrics:
- coderd_oauth2_external_requests_rate_limit_total
- coderd_api_workspace_latest_build_total

* Apply suggestions from code review

add link to follow-up issue

Co-authored-by: Cian Johnston <public@cianjohnston.ie>

---------

Co-authored-by: Cian Johnston <public@cianjohnston.ie>
2024-04-24 11:23:24 +03:00
Frederik Dudzik 5780050493 chore(docs): fix broken links (#13056)
* fix broken links

* fmt
2024-04-24 11:21:22 +03:00
Cian Johnston a04c76ce40 ci: release: test migrations before building (#13051) 2024-04-24 08:31:01 +01:00
Kayla Washburn-Love 215dd7b152 feat: show version on login page (#13033) 2024-04-23 11:18:56 -06:00
Mathias Fredriksson a69fc657f2 chore(coderd/database): reduce dbpurge load with smaller batches of agent stats (#13049) 2024-04-23 15:01:56 +03:00
Bruno Quaresma 2f7f9d022a refactor(site): reorganize template schedule settings form (#13031)
Close https://github.com/coder/coder/issues/12617

**Demo**

https://github.com/coder/coder/assets/3165839/66d4f238-d31f-4ee8-a3de-ce68215b0492

**Autostop**
![Image](https://github.com/coder/coder/assets/3165839/7d7430b9-fdb6-4842-ab2d-3b22cebe579e)

**Autostart**
![Image](https://github.com/coder/coder/assets/3165839/fd65865e-f996-4b17-b16b-679fd8c6b449)

**Dormancy**
![Image](https://github.com/coder/coder/assets/3165839/625e4769-7742-47c7-bce8-b33a54abaa34)
2024-04-23 08:59:19 -03:00
Cian Johnston e57ca3cdaa feat(scripts): add script to check schema between migrations (#13037)
- migrations: allow passing in a custom migrate.FS
- gen/dump: extract some functions to dbtestutil
- scripts: write script to test migrations
2024-04-23 12:43:14 +01:00
dependabot[bot] 81fcdf717b chore: bump github.com/gohugoio/hugo from 0.124.0 to 0.125.2 (#13024)
* chore: bump github.com/gohugoio/hugo from 0.124.0 to 0.125.2

Bumps [github.com/gohugoio/hugo](https://github.com/gohugoio/hugo) from 0.124.0 to 0.125.2.
- [Release notes](https://github.com/gohugoio/hugo/releases)
- [Changelog](https://github.com/gohugoio/hugo/blob/master/hugoreleaser.toml)
- [Commits](https://github.com/gohugoio/hugo/compare/v0.124.0...v0.125.2)

---
updated-dependencies:
- dependency-name: github.com/gohugoio/hugo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* add license to allowlist

* syntax

* wrong format

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jon Ayers <jon@coder.com>
2024-04-23 02:27:32 +00:00
Kayla Washburn-Love fab5591cf6 chore: change site_configs.value to text (#13036)
* chore: change `site_configs.value` to `text`

* `make gen`
2024-04-22 17:25:36 -06:00
Kyle Carberry d3f3ace220 chore: reduce dashboard requests from seeded data (#13034)
* chore: reduce requests the dashboard makes from seeded data

We already inject all of this content in `index.html`.

There was also a bug with displaying a loading indicator when
the workspace proxies endpoint 404s.

* Fix first user fetch

* Add util

* Add cached query for entitlements and experiments

* Fix authmethods unnecessary request

* Fix unnecessary region request

* Fix fmt

* Debug

* Fix test
2024-04-22 16:07:56 -04:00
Kyle Carberry 8d1220e0c8 chore: add generate script for azure instance identity (#13028)
* chore: add generate script for azure instance identity

This also adds new issuing certificates from:
https://learn.microsoft.com/en-us/azure/security/fundamentals/azure-ca-details?tabs=certificate-authority-chains

* Fix shell lint

* Fix shell fmt

* Fix RSA issuing certificate
2024-04-22 15:39:08 -04:00
Michael Brewer 7bd1b3bdb8 chore: fix broken mainline link (#13015)
* docs(releases): fix 404 for mainline link

* Delete x.sh

* Update releases.md

* docs: format and use term 'bleeding edge' for mainline releases
2024-04-22 10:57:59 -05:00
dependabot[bot] 3af317317a ci: bump crate-ci/typos from 1.19.0 to 1.20.9 in the github-actions group (#13027)
* ci: bump crate-ci/typos in the github-actions group

Bumps the github-actions group with 1 update: [crate-ci/typos](https://github.com/crate-ci/typos).


Updates `crate-ci/typos` from 1.19.0 to 1.20.9
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.19.0...v1.20.9)

---
updated-dependencies:
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add `pn` exclusion

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kyle Carberry <kyle@carberry.com>
2024-04-22 10:25:23 -04:00
dependabot[bot] 2e49fa94d4 chore: bump github.com/coder/terraform-provider-coder (#13022)
Bumps [github.com/coder/terraform-provider-coder](https://github.com/coder/terraform-provider-coder) from 0.20.1 to 0.21.0.
- [Release notes](https://github.com/coder/terraform-provider-coder/releases)
- [Changelog](https://github.com/coder/terraform-provider-coder/blob/main/.goreleaser.yml)
- [Commits](https://github.com/coder/terraform-provider-coder/compare/v0.20.1...v0.21.0)

---
updated-dependencies:
- dependency-name: github.com/coder/terraform-provider-coder
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-22 08:23:35 -04:00
dependabot[bot] ea472c5388 chore: bump google.golang.org/api from 0.172.0 to 0.175.0 (#13026)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.172.0 to 0.175.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.172.0...v0.175.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-22 08:23:22 -04:00
Michael Brewer d2acb6776e chore: fix link to install (#13019) 2024-04-22 08:23:09 -04:00
Mathias Fredriksson 3adcccb618 fix(coderd/database): reduce db load via dbpurge advisory locking (#13021) 2024-04-22 11:10:32 +00:00
Aaron Lehmann 8a1216254e feat(cli): add --env flag for coder ssh (#12991)
This allows environment variables to be set on the SSH session.

Example:

   coder ssh myworkspace --env VAR1=val1,VAR2=val2
2024-04-22 13:13:48 +03:00
Mathias Fredriksson e17e8aa3c9 feat(coderd/database): keep only 1 day of workspace_agent_stats after rollup (#12674) 2024-04-22 13:11:50 +03:00
Michael Brewer 4a6693a171 chore: fix 404 for managed terraform variables (#13018) 2024-04-22 13:09:05 +03:00
Frederik Dudzik b40f54f603 chore(docs): make external auth docs easier to follow (#12970)
* add additional context to github external auth provider documentation

* Apply suggestions from code review

Co-authored-by: Kyle Carberry <kyle@carberry.com>

* Update docs/admin/external-auth.md

* fmt

* fmt

---------

Co-authored-by: Kyle Carberry <kyle@carberry.com>
2024-04-20 16:26:53 +00:00
Marcin Tojek 3d7740bd32 test(site): add e2e tests for workspace proxies (#13009) 2024-04-19 14:45:52 +02:00
Colin Adler 3aa0d73811 chore: fix down migration 196 (#13006)
It didn't account for null values.
2024-04-18 18:47:02 -05:00
Danny Kopping 319fd5bf1d chore: add e2e test against an external auth provider during workspace creation (#12985) 2024-04-18 19:43:10 +02:00
Marcin Tojek 75223dfd8b test(site): add e2e tests for observability 2024-04-18 12:50:34 +02:00
Ben Potter f5a32b3f27 docs: explain that mainline stays around for one month now (#12993)
* docs: mainline stays around for one month

* switch to .x to encourage latest
2024-04-17 20:10:40 -05:00
Dean Sheather d426569d4a fix: make terminal raw in ssh command on windows (#12990) 2024-04-17 18:01:20 +00:00
Mathias Fredriksson 92190443ff fix(coderd/metricscache): avoid logging error for no rows (#12988)
Fixes #12938
2024-04-17 20:43:13 +03:00
Colin Adler 6b4eb03192 chore: give additional time in tests for tailnetAPIConnector graceful disconnect (#12980)
Failure seen here: https://github.com/coder/coder/actions/runs/8711258577/job/23894964182?pr=12979
2024-04-17 12:38:17 -05:00
dependabot[bot] 3338cdca77 chore: bump github.com/moby/moby (#12960)
Bumps [github.com/moby/moby](https://github.com/moby/moby) from 25.0.2+incompatible to 26.0.1+incompatible.
- [Release notes](https://github.com/moby/moby/releases)
- [Commits](https://github.com/moby/moby/compare/v25.0.2...v26.0.1)

---
updated-dependencies:
- dependency-name: github.com/moby/moby
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-17 13:30:53 -04:00
Kyle Carberry 227e632053 fix: add grace period before showing replicas license error (#12989)
Fixes #8665.
2024-04-17 13:30:32 -04:00
Danny Kopping b85d5d8491 feat: add warning about use of old/removed/invalid experiments (#12962) 2024-04-17 16:59:31 +02:00
Marcin Tojek cb8c576c93 test(site): add e2e tests for network (#12987) 2024-04-17 16:06:49 +02:00
Marcin Tojek ee7dda8111 refactor(site): verify deployment config flags in e2e tests (#12986) 2024-04-17 11:51:55 +00:00
Jon Ayers 0c993566dd hotfix: skip dependency license review on main (#12982) 2024-04-16 23:08:22 -05:00
Jon Ayers 80f5978124 chore: add license review to CI (#12981) 2024-04-16 22:52:07 -05:00
Colin Adler 777dfbe965 feat(enterprise): add ready for handshake support to pgcoord (#12935) 2024-04-16 15:01:10 -05:00
Colin Adler 942e90270e fix: disable azureidentity test on darwin (#12979)
See https://github.com/coder/coder/issues/12978
2024-04-16 19:33:04 +00:00
Colin Adler 231fc26c92 fix(coderd): properly calculate query latency for tailnet queries (#12944)
The defer used seems correct, but the `time.Since` will always resolve
immediately since it's a param to the deferred function.
2024-04-16 19:03:27 +00:00
Colin Adler ba52a4fbe2 chore: fix linting issue (#12945)
The error wasn't used.
2024-04-16 13:50:46 -05:00
Cian Johnston 8e1e0f04a4 feat(cli): support bundle: show links to docs/admin/healthcheck (#12974) 2024-04-16 16:21:09 +01:00
Marcin Tojek b598aef543 test(site): add e2e tests for user auth (#12971) 2024-04-16 14:32:07 +02:00
Cian Johnston 407e61ecd4 feat(cli): support bundle: dump healthcheck summary (#12963)
* refactor(codersdk): extract common fields from HealthReport and friends
* feat(codersdk/healthsdk): add Summarize() method
* feat(cli): support bundle: dump healthcheck summary
2024-04-16 13:31:56 +01:00
Danny Kopping 06e042acfa chore: apply shellcheck recommendation which was causing "make lint" to fail locally (#12972) 2024-04-16 13:06:08 +02:00
Kyle Carberry 41ca6e4f7f chore: add created_at to workspace resource telemetry (#12969) 2024-04-15 20:06:59 +00:00
Garrett Delfosse 3ab5a51ec2 feat: add listening ports protocol selector (#12915) 2024-04-15 15:00:24 -04:00
Marcin Tojek 49689162bb test(site): add e2e tests for security (#12961) 2024-04-15 19:31:33 +02:00
Cian Johnston 9a4703a311 feat(coderd/healthcheck): improve detection of STUN issues (#12951)
Adds checks to coderd/healthcheck/derphealth for STUN issues:
- Alerts if there is not least one healthy STUN server,
- Alerts if we see variable port mapping.
2024-04-15 17:10:49 +01:00
Stephen Kirby c13909a1a2 chore: fix broken links in the jfrog guide (#12835)
* replaced jfrog guide links

* replaced github link

* fmt

---------

Co-authored-by: Muhammad Atif Ali <atif@coder.com>
2024-04-15 09:51:58 -05:00
Steven Masley d9da054c9d chore: update generated array type definitions in TypeScript to be readonly (#12947)
* chore: types generated handling readonly slices

* add -update flag to update goldens

* revert excess gens

* fix: update most UI types to account for readonly modifiers

* fix: remove accidental mutation from NavBarView

* fix: remove mutation warning for BatchUpdateConfirmation stories

* fix: remove mutation warning for BactchUpdateConfirmation

* fix: format ActiveUserChart

* fix: update import to make linter happy

* fix: update fmt issue

* fix: disable file write lint rule from unit test

---------

Co-authored-by: Parkreiner <throwawayclover@gmail.com>
2024-04-15 09:46:10 -04:00
Kayla Washburn-Love 7cf8577f1c label some template settings as enterprise (#12952) 2024-04-15 09:24:11 -04:00
Kyle Carberry d3790bb5be fix: use provided username when fetching workspaces (#12955) 2024-04-13 14:39:57 -04:00
Kayla Washburn-Love 00fcf36999 test: add an e2e audit logs test (#12868) 2024-04-12 14:01:54 -06:00
Marcin Tojek cf2d2a98bd test(site): add e2e tests for appearance (#12950) 2024-04-12 14:46:44 +02:00
Cian Johnston b71af32113 chore(docs): add support bundle guide (#12931)
Adds a guide explaining support bundles.
2024-04-12 10:11:05 +01:00
Marcin Tojek dcf1d3a9ae test(site): add e2e tests for experiments (#12940) 2024-04-12 10:42:27 +02:00
Cian Johnston b163bc7f01 fix(support): correctly rename existing agent connection info, add real netcheck (#12946) 2024-04-12 09:40:04 +01:00
Kayla Washburn-Love c5367c201b test: fix url checks in e2e tests (#12881) 2024-04-11 15:48:53 -06:00
Steven Masley 93b46fe1f6 chore: skip global.setup if first user already exists (#12930)
* chore: skip global.setup if first user already exists

treat test as a setup, rather than a test

Co-authored-by: Kayla Washburn-Love <mckayla@hey.com>

---------

Co-authored-by: Kayla Washburn-Love <mckayla@hey.com>
2024-04-11 21:10:40 +00:00
Kayla Washburn-Love 2ad7fcc0b7 fix: show template autostop setting when it overrides the workspace setting (#12910) 2024-04-11 13:08:51 -06:00
Steven Masley 22785a307c chore: add -agpl to agpl e2e artifacts (#12943)
* chore: -agpl added to agpl e2e artifacts

Before was doing 'false' at the end of artifacts
2024-04-11 16:57:40 +00:00
Steven Masley b9936a4671 chore: deconflict e2e enterprise and AGPL artifacts in ci (#12941) 2024-04-11 09:42:21 -05:00
Cian Johnston fad97a14f9 fix(cli): allow generating partial support bundles with no workspace or agent (#12933)
* fix(cli): allow generating partial support bundles with no workspace or agent

* nolint control flag
2024-04-11 10:09:10 +01:00
Spike Curtis a231b5aef5 feat: add src_id and dst_id indexes to tailnet_tunnels (#12911)
Fixes #12780

Adds indexes to the `tailnet_tunnels` table to speed up `GetTailnetTunnelPeerIDs` and `GetTailnetTunnelPeerBindings` queries, which match on `src_id` and `dst_id`.
2024-04-11 10:05:53 +04:00
Stephen Kirby ab116af543 added releases.md to manifest (#12936) 2024-04-10 18:31:21 -05:00
Steven Masley 8da8b89af7 test: verify actually uploaded license with assert (#12934)
Prior page.GetByText did not assert it existed
2024-04-10 18:02:08 -05:00
Colin Adler e801e878ba feat: add agent acks to in-memory coordinator (#12786)
When an agent receives a node, it responds with an ACK which is relayed
to the client. After the client receives the ACK, it's allowed to begin
pinging.
2024-04-10 17:15:33 -05:00
Kayla Washburn-Love 9cf2358114 ci: execute enterprise and non-enterprise e2e tests concurrently (#12872) 2024-04-10 15:42:53 -06:00
Steven Masley 7fd9a75ad9 chore: nix shell to support playwright e2e tests (#12917)
* chore: nix shell to support playwright e2e tests

nix is running an older version of chromium, so had to reduce the
playwright version.

* Add to e2e readme

* add enterprise test comment

* add note about install to readme

* make fmt

* remove shellhook message

Co-authored-by: Kayla Washburn-Love <mckayla@hey.com>

* add link to nixos playwright package to get version

* formatting

---------

Co-authored-by: Kayla Washburn-Love <mckayla@hey.com>
2024-04-10 14:08:25 -05:00
Steven Masley 566f8f231d chore: add unit test for pass through external auth query params (#12928)
* chore: verify pass through external auth query params

Unit test added to verify behavior of query params set in the
auth url for external apps. This behavior is intended to specifically
support Auth0 audience query param.
2024-04-10 13:58:29 -05:00
Spike Curtis 06eae954c9 fix: stop sending DeleteTailnetPeer when coordinator is unhealthy (#12925)
fixes #12923

Prevents Coordinate peer connections from generating spurious database queries like DeleteTailnetPeer when the coordinator is unhealthy.

It does this by checking the health of the querier before accepting a connection, rather than unconditionally accepting it only for it to get swatted down later.
2024-04-10 22:49:13 +04:00
Steven Masley a607d5610e chore: disable pgcoord (HA) when --in-memory (#12919)
* chore: disable pgcoord (HA) when --in-memory

HA does not make any sense while using in-memory database
2024-04-10 11:05:55 -05:00
Steven Masley 838e8df5be chore: merge apikey/token session config values (#12817)
* chore: merge apikey/token session config values

There is a confusing difference between an apikey and a token. This
difference leaks into our configs. This change does not resolve the
difference. It only groups the config values to try and manage any
bloat that occurs from adding more similar config values
2024-04-10 10:34:49 -05:00
Steven Masley 4dc293d930 chore: add date information to windows startup logs (#12905) 2024-04-10 09:41:05 -05:00
Marcin Tojek e266ecf91b test(site): fix flaky outdated agent test (#12927) 2024-04-10 16:09:44 +02:00
Garrett Delfosse acaa254099 feat: link with protocol on shared ports (#12908) 2024-04-10 09:29:24 -04:00
Marcin Tojek 2f2a395ba9 e2e tests for deployment/licenses (#12926) 2024-04-10 15:00:39 +02:00
Marcin Tojek b6359b0a89 fix: ignore gomock temporary files (#12924) 2024-04-10 08:48:56 +00:00
Spike Curtis 5469011018 fix: stop logging session shutdown as warning (#12922)
A customer hit like 200k of ErrSessionShutdown, which just dupes any errors we would have generated when shutting down the session for e.g. Ping failures.
2024-04-10 11:50:46 +04:00
Steven Masley 0a8c8ce5cc chore: remove InsertWorkspaceAgentStat query (#12869)
* chore: remove InsertWorkspaceAgentStat query

InsertWorkspaceAgentStats (batch) exists. We only used the singular in
a single unit test place. Removing the single for the batch, reducing
the interface size.
2024-04-09 12:35:27 -05:00
Garrett Delfosse 1d4bf30c0d feat: add s suffix to use HTTPS for ports (#12862) 2024-04-09 12:06:22 -04:00
Steven Masley 189b8626d0 chore: deprecate agent report-stats endpoint (#12880)
* chore: deprecate agent report-stats endpoint

Agent API is now used instead.

* Update coderd/workspaceagents.go

Co-authored-by: Spike Curtis <spike@coder.com>

---------

Co-authored-by: Spike Curtis <spike@coder.com>
2024-04-09 09:38:26 -05:00
Marcin Tojek 08451ce80c feat: remove health link from deployment sidebar (#12914) 2024-04-09 13:47:47 +01:00
Mathias Fredriksson 0178bfe134 fix(examples): copy /etc/skel on init in docker template (#12913)
Fixes #10209
2024-04-09 14:54:17 +03:00
Marcin Tojek 28754a79e5 docs: describe air-gapped architecture (#12897) 2024-04-09 12:33:06 +02:00
coryb d82f2fd416 fix: update typo in audit log field (#12907) 2024-04-08 13:57:38 -05:00
dependabot[bot] 7179c86df3 chore: bump golang.org/x/oauth2 from 0.18.0 to 0.19.0 (#12893)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.18.0 to 0.19.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 11:52:35 -05:00
dependabot[bot] 11123018a2 chore: bump google.golang.org/grpc from 1.62.1 to 1.63.0 (#12892)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.62.1 to 1.63.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.62.1...v1.63.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 15:57:52 +00:00
dependabot[bot] 589434e8d8 chore: bump golang.org/x/tools from 0.19.0 to 0.20.0 (#12890)
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.19.0 to 0.20.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.19.0...v0.20.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:46:16 -05:00
dependabot[bot] 9a7d8034cb chore: bump golang.org/x/net from 0.22.0 to 0.24.0 (#12888)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.22.0 to 0.24.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.24.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:31:09 -05:00
dependabot[bot] f99fd807b1 chore: bump golang.org/x/sync from 0.6.0 to 0.7.0 (#12895)
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.6.0 to 0.7.0.
- [Commits](https://github.com/golang/sync/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:30:47 -05:00
dependabot[bot] 8ba8ec2f19 chore: bump github.com/elastic/go-sysinfo from 1.13.1 to 1.14.0 (#12894)
Bumps [github.com/elastic/go-sysinfo](https://github.com/elastic/go-sysinfo) from 1.13.1 to 1.14.0.
- [Release notes](https://github.com/elastic/go-sysinfo/releases)
- [Commits](https://github.com/elastic/go-sysinfo/compare/v1.13.1...v1.14.0)

---
updated-dependencies:
- dependency-name: github.com/elastic/go-sysinfo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:30:32 -05:00
dependabot[bot] 24135a2d0f chore: bump golang.org/x/term from 0.18.0 to 0.19.0 (#12886)
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.18.0 to 0.19.0.
- [Commits](https://github.com/golang/term/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: golang.org/x/term
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 15:27:46 +03:00
Spike Curtis 3b7380fa00 fix: fix race in assertWorkspaceLastUsedAtUpdated (#12899)
fixes #12789

Stats are collected asynchronously with respect to sessions ending.  Flush repeatedly so that we pick up the collection if we missed it.
2024-04-08 16:22:33 +04:00
Garrett Delfosse f96ce80ab9 feat: add owner groups to workspace data (#12841) 2024-04-05 15:06:17 -04:00
Kayla Washburn-Love c4b26f335a test: verify that enterprise tests are being run (#12871) 2024-04-05 11:45:32 -06:00
Colin Adler a2b28f80d7 fix(coderd): prevent agent reverse proxy from using HTTP[S]_PROXY envs (#12875)
Updates https://github.com/coder/coder/issues/12790
2024-04-05 12:29:08 -05:00
Mathias Fredriksson b06452ee88 fix(install.sh): remove extracted files after installation (#12879) 2024-04-05 19:04:12 +03:00
Marcin Tojek 7c0fac9906 docs: describe devcontainers as deployment model (#12877) 2024-04-05 15:30:49 +02:00
Mathias Fredriksson c243210ae5 fix(install.sh): change post-install advisory when installing specific version (#12878) 2024-04-05 15:55:11 +03:00
Michael Brewer 61e5721caa fix(install.sh): use --version when provided (#12873) 2024-04-05 13:14:49 +03:00
Bruno Quaresma 3fbcdb0ddc chore(site): add e2e tests for groups (#12866) 2024-04-04 21:56:28 -03:00
Kayla Washburn-Love bc9ea61eb4 ci: disable enterprise e2e tests temporarily (#12874) 2024-04-04 17:39:07 -06:00
Marcin Tojek 90efa1b846 docs: describe multi-cloud architecture (#12857) 2024-04-04 15:42:26 +02:00
Bruno Quaresma 41b8ff3e81 chore(site): add e2e to test add and remove user (#12851) 2024-04-04 09:21:03 -03:00
Stephen Kirby a7234f61a1 chore: update change log to v2.10.0 and install docs for release channels (#12863)
* 2.10.0 changelog

* updated install docs for mainline/stable releases

* make fmt

* cpp icon -> C++

* added disclaimer on MAX_TTL, support bundle info

* 'release schedule'

* lowercase mainline

* Agent OOM protection info

* minor tweak
2024-04-03 16:43:49 -05:00
Stephen Kirby bf19e3469f added 'JFrog' in front of XRay in guide (#12860) 2024-04-03 14:44:57 -05:00
Stephen Kirby d9211b6693 chore(docs): replace FAQ twisties with h3s (#12859)
* replace FAQ twisties with h3s

* make fmt
2024-04-03 14:44:28 -05:00
Colin Adler cb6fea61df chore: upgrade go to 1.21.9 (#12861) 2024-04-03 13:20:26 -05:00
Steven Masley a3187dc30f chore: enforce unique linked_ids (#12815)
* chore: enforce unique linked_ids

Duplicate linked_ids make no sense. 2 users cannot share the same
source user from a provider
2024-04-03 13:17:11 -05:00
Mathias Fredriksson 65f8d18ce5 feat(install.sh): add support for --mainline (default) and --stable (#12858)
Fixes #12461
2024-04-03 20:26:48 +03:00
Jon Ayers 426e9f2b96 feat: support adjusting child proc oom scores (#12655) 2024-04-03 09:42:03 -05:00
Marcin Tojek ac8d1c6696 docs: describe single region and multi-region deployments (#12779) 2024-04-03 12:45:01 +02:00
Kayla Washburn-Love caa49ea6a1 chore: stabilize light theme (#12855) 2024-04-02 16:06:31 -06:00
Colin Adler 41914256b3 chore: update terraform version in install.sh (#12856) 2024-04-02 16:53:36 -05:00
Kayla Washburn-Love 1dd840d149 test: add an e2e test for removing a group (#12844) 2024-04-02 11:29:43 -06:00
Kayla Washburn-Love f705f9a5eb test: ensure RequireActiveVersion is actually set when testing with AGPL store (#12843) 2024-04-02 11:29:22 -06:00
Kyle Carberry 7698cfda72 chore: remove unnecessary extraction library (#12847)
This was allocating ~256KB on init.
2024-04-02 11:19:54 -04:00
Steven Masley b5b5c37d03 docs: describe mutually exclusive create workspace template fields (#12834)
* docs: describe mutually exclusive create workspace template fields

Ideally we could do this in the OpenAPI spec, but there is no first
class "mutually exclusive" feature in OpenAPI. So in lieu of something
more complex, or changing our struct/validation, a description comment
should suffice.

* chore: Add description to code sample as well
2024-04-02 10:11:24 -05:00
Steven Masley 5137433123 chore: add validation errors to the cli output (#12814)
* chore: add validation errors to the cli output
2024-04-02 10:02:30 -05:00
Kyle Carberry 94e82f9662 chore: use fork of chroma to remove unused inits (#12842)
* chore: use fork of chroma to remove unused inits

This seems fine to do since compilation errors would occur
if it were actually in use.

Everything seems fine here.

* Update validator
2024-04-02 14:14:38 +00:00
Danny Kopping 79fb8e43c5 feat: expose workspace statuses (with details) as a prometheus metric (#12762)
Implements #12462
2024-04-02 09:57:36 +02:00
Toshiki Shimomura 114830de26 Fix coder-logstream-kube typo in deployment-logs.md (#12845) 2024-04-02 03:36:35 +00:00
Kyle Carberry f5a70500d2 chore: update tailscale for to lazily load hostinfo (#12840)
Speeds up `init()` by ~10ms
2024-04-01 17:04:39 -04:00
dependabot[bot] b47fb41783 chore: bump github.com/bramvdbogaerde/go-scp from 1.3.0 to 1.4.0 (#12825)
Bumps [github.com/bramvdbogaerde/go-scp](https://github.com/bramvdbogaerde/go-scp) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/bramvdbogaerde/go-scp/releases)
- [Commits](https://github.com/bramvdbogaerde/go-scp/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/bramvdbogaerde/go-scp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 20:48:52 +00:00
Kyle Carberry 2a30194ed7 chore: use validator fork to fix 10ms of init time (#12837)
@ammario inspired me
2024-04-01 16:31:26 -04:00
Kyle Carberry d428c05694 chore: move log output message before logs begin streaming (#12836) 2024-04-01 20:02:50 +00:00
Bruno Quaresma 7c1d10b952 chore(site): upgrade storybook to v8 (#12831) 2024-04-01 16:12:17 -03:00
Spike Curtis 3addf7ac5d fix: use latest coder/tailscale
Bad merge in #12252 clobbered #12574
2024-04-01 22:42:38 +04:00
dependabot[bot] 12ecc6554c chore: bump google.golang.org/api from 0.171.0 to 0.172.0 (#12827)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.171.0 to 0.172.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.171.0...v0.172.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 18:14:17 +00:00
dependabot[bot] a4be2831d6 chore: bump github.com/charmbracelet/glamour from 0.6.0 to 0.7.0 (#12824)
Bumps [github.com/charmbracelet/glamour](https://github.com/charmbracelet/glamour) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/charmbracelet/glamour/releases)
- [Commits](https://github.com/charmbracelet/glamour/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: github.com/charmbracelet/glamour
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 18:13:51 +00:00
dependabot[bot] 3e28250849 chore: bump github.com/cenkalti/backoff/v4 from 4.2.1 to 4.3.0 (#12826)
Bumps [github.com/cenkalti/backoff/v4](https://github.com/cenkalti/backoff) from 4.2.1 to 4.3.0.
- [Commits](https://github.com/cenkalti/backoff/compare/v4.2.1...v4.3.0)

---
updated-dependencies:
- dependency-name: github.com/cenkalti/backoff/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 13:02:23 -05:00
dependabot[bot] b7f5456e35 ci: bump the github-actions group with 1 update (#12828)
Bumps the github-actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action).


Updates `aquasecurity/trivy-action` from 0.18.0 to 0.19.0
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](https://github.com/aquasecurity/trivy-action/compare/062f2592684a31eb3aa050cc61e7ca1451cecd3d...d710430a6722f083d3b36b8339ff66b32f22ee55)

---
updated-dependencies:
- dependency-name: aquasecurity/trivy-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-01 13:01:09 -05:00
Bruno Quaresma 2f437005b7 chore(site): clean up mocks after each test (#12805) 2024-04-01 13:14:36 -03:00
elasticspoon cfb94284e0 feat(cli): add golden tests for errors (#11588) (#12698)
* feat(cli): add golden tests for errors (#11588)

Creates golden files from `coder/cli/errors.go`.
Adds a unit test to test against golden files.
Adds a make file command to regenerate golden files.
Abstracts test against golden files.
2024-04-01 09:19:26 -05:00
Bruno Quaresma 75bf41ba02 chore(site): use static date and ignore dynamic values for storybook (#12830) 2024-04-01 11:02:54 -03:00
Kayla Washburn-Love 1d2d008b45 chore: add e2e tests for template permissions (#12731) 2024-03-29 10:43:23 -06:00
Steven Masley eeb3d63be6 chore: merge authorization contexts (#12816)
* chore: merge authorization contexts

Instead of 2 auth contexts from apikey and dbauthz, merge them to
just use dbauthz. It is annoying to have two.

* fixup authorization reference
2024-03-29 10:14:27 -05:00
Michael Brewer 8e2d026d99 docs: document how to run workspace-proxy as a system service (#12810)
* docs: document how to run workspace-proxy as a system service

* Update workspace-proxies.md

* Update workspace-proxies.md

Co-authored-by: Muhammad Atif Ali <me@matifali.dev>

* docs: fix duplication

---------

Co-authored-by: Muhammad Atif Ali <me@matifali.dev>
2024-03-29 15:06:32 +00:00
Kayla Washburn-Love f3cfe10c26 chore: add more e2e template settings tests (#12717) 2024-03-28 16:00:27 -06:00
Steven Masley 8cf1e84bb5 chore: ensure root handler has sudomain app mw (#12812)
Enterprise routes like scim touch this.
2024-03-28 15:49:43 -05:00
Steven Masley b785e996f8 chore: explain GIT_ASKPASS behavior in docs (#12784)
* chore: docs explaining GIT_ASKPASS behavior

- VSCode configuration requirements
2024-03-28 13:59:03 -05:00
Mathias Fredriksson 79441e3609 perf(coderd/database): optimize GetWorkspaceAgentAndLatestBuildByAuthToken (#12809) 2024-03-28 19:38:16 +02:00
Steven Masley 93a233ac10 chore: write auto-update message after success (#12804) 2024-03-28 08:55:15 -05:00
Mathias Fredriksson d50c20c453 fix(coderd/database): add fk index for workspace_agent_scripts (#12791) 2024-03-28 14:31:58 +02:00
Danny Kopping d734f3fb74 chore: reduce azure CA cert validity check period to 2 months (#12788)
Signed-off-by: Danny Kopping <danny@coder.com>
2024-03-28 11:17:02 +02:00
Muhammad Atif Ali 0288e73e9b docs: add guide for Xray integration (#12629)
* docs: add guides for Xray integration

* `make fmt`
2024-03-28 04:48:50 +03:00
Colin Adler dc8cf3eea5 fix: nil ptr dereference when removing a license (#12785) 2024-03-27 15:59:35 -05:00
Muhammad Atif Ali 5235faa79f chore(site): remove max ttl from scheduling description (#12715) 2024-03-27 14:24:26 -05:00
Mathias Fredriksson 539d6b0f3b test(coderd): fix template name too long in TestPatchTemplateMeta (#12781) 2024-03-27 18:25:42 +02:00
Mathias Fredriksson 421bf7e785 fix(coderd): use insights for DAUs, simplify metricscache (#12775)
Fixes #12134
Fixes https://github.com/coder/customers/issues/384
Refs #12122
2024-03-27 18:10:14 +02:00
Bruno Quaresma 5d82a78d4c fix(site): fix and improve pending state on template editor UI (#12766) 2024-03-27 12:42:07 -03:00
Mathias Fredriksson 47fd190064 fix(coderd/database): improve perf of GetTemplateInsightsByInterval (#12773)
Refs #12122
2024-03-27 14:10:46 +02:00
Mathias Fredriksson ba1eaceda4 feat(coderd): add sftp to insights apps (#12675) 2024-03-27 14:09:29 +02:00
Danny Kopping 6cb1fc8956 chore: add note about options use in numeric parameters (#12770) 2024-03-27 13:46:41 +02:00
Mathias Fredriksson 0da29d74ac fix(coderd/database): improve query perf of GetTemplateAppInsights (#12767)
Refs #12122
2024-03-27 12:28:36 +02:00
Danny Kopping a74ef4096e feat: allow number options with monotonic validation (#12726)
NOTE: terraform-provider-coder was updated to facilitate this change, and your template will require v0.19.0 for this feature to work. You can run terraform init -upgrade in your template directory. If you have a version constraint set, ensure it points to this version.
2024-03-27 08:54:42 +00:00
Ammar Bandukwala 0d9010e150 chore: fix 30% startup time hit from userpassword (#12769)
pbkdf2 is too expensive to run in init, so this change makes it load
lazily. I introduced a lazy package that I hope to use more in my
`GODEBUG=inittrace=1` adventure.


Benchmark results:

```
$ hyperfine "coder --help" "coder-new --help"
Benchmark 1: coder --help
  Time (mean ± σ):      82.1 ms ±   3.8 ms    [User: 93.3 ms, System: 30.4 ms]
  Range (min … max):    72.2 ms …  90.7 ms    35 runs
 
Benchmark 2: coder-new --help
  Time (mean ± σ):      52.0 ms ±   4.3 ms    [User: 62.4 ms, System: 30.8 ms]
  Range (min … max):    41.9 ms …  62.2 ms    52 runs
 
Summary
  coder-new --help ran
    1.58 ± 0.15 times faster than coder --help
```
2024-03-26 20:47:14 -05:00
dependabot[bot] 73fbdbbe2d chore: bump github.com/coreos/go-oidc/v3 from 3.9.0 to 3.10.0 (#12739)
Bumps [github.com/coreos/go-oidc/v3](https://github.com/coreos/go-oidc) from 3.9.0 to 3.10.0.
- [Release notes](https://github.com/coreos/go-oidc/releases)
- [Commits](https://github.com/coreos/go-oidc/compare/v3.9.0...v3.10.0)

---
updated-dependencies:
- dependency-name: github.com/coreos/go-oidc/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-26 12:45:42 -05:00
Colin Adler 4d5a7b2d56 chore(codersdk): move all tailscale imports out of codersdk (#12735)
Currently, importing `codersdk` just to interact with the API requires
importing tailscale, which causes builds to fail unless manually using
our fork.
2024-03-26 12:44:31 -05:00
dependabot[bot] 0bea8906d4 chore: bump google.golang.org/api from 0.170.0 to 0.171.0 (#12737)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.170.0 to 0.171.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.170.0...v0.171.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-26 17:14:01 +00:00
dependabot[bot] a323e30450 chore: bump github.com/aws/aws-sdk-go-v2 from 1.25.3 to 1.26.0 (#12738)
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.25.3 to 1.26.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.25.3...v1.26.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-26 12:03:01 -05:00
Mathias Fredriksson ae0ee622bb fix(coderd/database): improve data exclusion in UpsertTemplateUsageStats (#12764)
The PostgreSQL query analyzer wasn't able to eliminate the agent stats without re-introducing this filter.

Before: https://explain.dalibo.com/plan/21h7gb4f4bef391g
After: https://explain.dalibo.com/plan/721ec1cccee91egc
2024-03-26 17:21:05 +02:00
Mathias Fredriksson f418ece9ae test(cli): prevent flake due to outdated build in TestSSH (#12760)
Fixes #12752
2024-03-26 10:46:58 +00:00
Spike Curtis 51491fc01b fix(provisionersdk): change test to use bash script instead of binary echo (#12759)
Just upgraded to macOS 14.4 and TestAgentScript/Run fails for me with error `signal: killed`.  I opened the test directory in a terminal and sure enough, when you execute the `echo` binary, it is immediately killed.  The binary has no extended attributes and is byte-identical to the one in `/bin/`.

This fix uses a different strategy: instead of copying the `echo` binary from the system around, we just copy a small bash script that _calls_ the `echo` command.
2024-03-26 14:37:20 +04:00
Colin Adler 5f28220eec fix(coderd): add timeout to websocket waitgroup on shutdown (#12754) 2024-03-26 03:04:15 +00:00
Kayla Washburn-Love cfb484fa25 fix: always use bash when executing web terminal tests (#12755) 2024-03-25 16:58:07 -06:00
dependabot[bot] 064a08efa5 ci: bump the github-actions group with 1 update (#12743)
Bumps the github-actions group with 1 update: [contributor-assistant/github-action](https://github.com/contributor-assistant/github-action).


Updates `contributor-assistant/github-action` from 2.3.1 to 2.3.2
- [Release notes](https://github.com/contributor-assistant/github-action/releases)
- [Commits](https://github.com/contributor-assistant/github-action/compare/v2.3.1...v2.3.2)

---
updated-dependencies:
- dependency-name: contributor-assistant/github-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-25 23:40:35 +03:00
Asher 40e5ad5499 feat: make OAuth2 provider not enterprise-only (#12732) 2024-03-25 11:52:22 -08:00
Garrett Delfosse 60f335113c chore: add protocol lag to shared ports description (#12728) 2024-03-25 15:28:38 -04:00
Kyle Carberry fd8010c26d chore: make build only run on main (#12753) 2024-03-25 19:15:18 +00:00
Kayla Washburn-Love 541ccd940c chore: change e2e testing port (#12751) 2024-03-25 13:07:34 -06:00
Kyle Carberry 03ab37b343 chore: remove middleware to request version and entitlement warnings (#12750)
This cleans up `root.go` a bit, adds tests for middleware HTTP transport
functions, and removes two HTTP requests we always always performed previously
when executing *any* client command.

It should improve CLI performance (especially for users with higher latency).
2024-03-25 15:01:42 -04:00
Cian Johnston ba3879ac47 fix(cli): fix newline escape sequence in support blurb (#12749) 2024-03-25 16:51:48 +00:00
Marcin Tojek 1e0bbd5e10 docs: describe operational readiness (#12723) 2024-03-25 17:10:24 +01:00
Mathias Fredriksson 7e183db199 test(coderd): fix todo for increased accuracy in insights test (#12727)
This PR updates the tests in `insights_test.go` to enable commented-out scenarios. This behavior was fixed by previous PRs in this stack. Note that the updated golden files are correct since they are "second template only" meaning that the newly introduced data is considered as expected. In other golden files there is no change since "only count once" is applied.
2024-03-25 17:55:53 +02:00
Mathias Fredriksson b183236482 feat(coderd/database): use template_usage_stats in *ByTemplate insights queries (#12668)
This PR updates the `*ByTempalte` insights queries used for generating Prometheus metrics to behave the same way as the new rollup query and re-written insights queries that utilize the rolled up data.
2024-03-25 17:42:02 +02:00
Cian Johnston 01f9a9ab77 feat(cli): unhide support bundle cmd (#12745)
* chore(cli): add another test to ensure no secret leakage

* feat(cli): unhide support bundle cmd
2024-03-25 15:14:27 +00:00
Mathias Fredriksson 2332d8197a feat(coderd/database): use template_usage_stats in GetUserActivityInsights query (#12672)
This PR updates the `GetUserActivityInsights` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 16:16:41 +02:00
Mathias Fredriksson a8ed689bda feat(coderd/database): use template_usage_stats in GetUserLatencyInsights query (#12671)
This PR updates the `GetUserLatencyInsights` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 16:07:40 +02:00
Mathias Fredriksson 5738a03930 feat(coderd/database): use template_usage_stats in GetTemplateAppInsights query (#12669)
This PR updates the `GetTemplateAppInsights` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 15:58:37 +02:00
Mathias Fredriksson 5f3be62c83 feat(coderd/database): use template_usage_stats in GetTemplateInsightsByInterval query (#12667)
This PR updates the `GetTemplateInsightsByInterval` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 15:45:49 +02:00
Mathias Fredriksson 35d08434a9 feat(coderd/database): use template_usage_stats in GetTemplateInsights query (#12666)
This PR updates the `GetTemplateInsights` query to use rolled up `template_usage_stats` instead of raw agent and app stats.
2024-03-25 15:33:31 +02:00
Mathias Fredriksson f34592f45d fix(coderd): skip logging error for cancelled query in agent report stats (#12730) 2024-03-25 12:20:16 +02:00
Jon Ayers 951dfaa99c feat: add workspace_id to workspace_build audit logs (#12718) 2024-03-22 15:10:32 -05:00
Kayla Washburn-Love 0966fe2560 fix: create workspace with optional auth providers (#12729) 2024-03-22 13:26:02 -06:00
Steven Masley c674128105 chore: allow search by build params in workspace search filter (#12694)
* chore: workspace search filter allow search by params
* has_param will return all workspaces with the param existance
* exact matching
2024-03-22 14:22:47 -05:00
Mathias Fredriksson b4fd819f0d test(coderd): enable dbrollup service for insights tests (#12673) 2024-03-22 20:18:20 +02:00
Mathias Fredriksson 12e6fbf11e feat(coderd/database): add dbrollup service to rollup insights (#12665)
Add `dbrollup` service that runs the `UpsertTemplateUsageStats` query
every 5 minutes, on the minute. This allows us to have fairly real-time
insights data when viewing "today".
2024-03-22 18:42:43 +02:00
Mathias Fredriksson 04f0510b09 feat(coderd/database): add template_usage_stats table and rollup query (#12664)
Add `template_usage_stats` table for aggregating tempalte usage data.
Data is rolled up by the `UpsertTemplateUsageStats` query, which fetches
data from the `workspace_agent_stats` and `workspace_app_stats` tables.
2024-03-22 18:33:34 +02:00
Kyle Carberry a6b8f381f0 feat: support windows containers in bootstrap script (#12662)
Fixes #7462.
2024-03-22 14:48:51 +00:00
Muhammad Atif Ali 58cbd8335f chore(site): reorganize template schedule strings page (#12714) 2024-03-22 15:44:16 +03:00
Marcin Tojek a7d9d87ba2 docs: use scale testing utility (#12643) 2024-03-22 11:33:31 +00:00
Colin Adler 37a05372fa fix: disable relay if built-in DERP is disabled (#12654)
Fixes https://github.com/coder/coder/issues/12493
2024-03-21 16:53:41 -05:00
Kyle Carberry d3c9aaf57b chore: update hero image for refreshed dashboard (#12712)
* chore: update hero image for refreshed dashboard

I used html.to.figma for this. It's in the following Figma:
https://www.figma.com/file/m4zkIU7e64BNyRlUShuluq/README-Screenshot?type=design&node-id=13-1993&mode=design&t=o2AhL8y7u8uFMVPI-0

I couldn't take screenshots directly, because I'm on Linux and the macOS
browser with the icons look much better.

* Try stacked images

* Try stacked inverted
2024-03-21 16:28:02 -04:00
Cian Johnston 28730ca3d8 fix(support): sanitize manifest (#12711) 2024-03-21 19:55:34 +00:00
Cian Johnston f2a9e515df feat(cli/support): confirm before creating bundle (#12684)
Forces user to confirm before creating a support bundle.
Also adds contextual information to the bundle under cli_logs.txt.
2024-03-21 17:06:28 +00:00
Cian Johnston 8ea5fb7115 chore(coderd/workspaceapps/apptest): fix test flake due to concurrent usage of same deployment (#12700)
- Updates existing tests under workspaceapps/apptest to not reuse existing appDetails as assertWorkspaceLastUsed(Not)?Updated calls FlushStats() which was causing racy test behaviour and incorrect test assertions.
- Expands scope of assertWorkspaceLastUsedAtUpdated and its counterpart to ProxySubdomain tests.
2024-03-21 15:38:38 +00:00
Cian Johnston 5454f4997b chore(ci): clean up databases after test finishes in CI (#12702) 2024-03-21 14:53:16 +00:00
Mathias Fredriksson 9c84fb7fb1 fix(coderd/agentapi): always write agent stats when provided (#12699) 2024-03-21 16:47:06 +02:00
Steven Masley bd6ad88077 chore: nolint always return error function (#12701) 2024-03-21 09:35:10 -05:00
Steven Masley b4492fffba chore: support multiple key:value search query params (#12690)
This more closely aligns with GitHub's label search style. Actual search params need to be converted to allow this format, by default they will throw an error if they do not support listing.
2024-03-21 08:37:19 -05:00
Bruno Quaresma 8499eacf67 chore(site): add tests for deprecate template flow (#12685)
Closes #12505
2024-03-21 10:37:08 -03:00
Steven Masley 131d0bd2ba chore: fix linting issue in main(#12697) 2024-03-20 20:15:01 -05:00
dependabot[bot] f93491ffe8 chore: bump github.com/docker/docker from 24.0.7+incompatible to 24.0.9+incompatible (#12692)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-21 02:14:30 +03:00
Dean Sheather 2b773f9034 fix: allow proxy version mismatch (with warning) (#12433) 2024-03-20 18:24:18 +00:00
Garrett Delfosse 4d9fe05f5a feat: add awsiamrds db auth driver (#12566) 2024-03-20 13:14:43 -04:00
Ammar Bandukwala 0d86dca852 fix(codersdk): abort in-progress writes/reads when closing websocket (#12650)
Fixes #9203

Related #12065

Also, adds some basic tracing infrastructure that we can build upon for more improvements.
2024-03-20 11:53:32 -05:00
Cian Johnston 92aa1eba97 fix(cli): port-forward: update workspace last_used_at (#12659)
This PR updates the coder port-forward command to periodically inform coderd that the workspace is being used:

- Adds workspaceusage.Tracker which periodically batch-updates workspace LastUsedAt
- Adds coderd endpoint to signal workspace usage
- Updates coder port-forward to periodically hit this endpoint
- Modifies BatchUpdateWorkspacesLastUsedAt to avoid overwriting with stale data

Co-authored-by: Danny Kopping <danny@coder.com>
2024-03-20 16:44:12 +00:00
Steven Masley d789a60d47 chore: remove max_ttl from templates (#12644)
* chore: remove max_ttl from templates

Completely removing max_ttl as a feature on template scheduling. Must use other template scheduling features to achieve autostop.
2024-03-20 10:37:57 -05:00
Bruno Quaresma d82e20152b feat(site): make listening ports scrollable (#12660) 2024-03-20 09:34:30 -03:00
Kayla Washburn-Love 9028717c9b fix: disable auto-create if external auth requirements aren't met (#12538) 2024-03-19 16:42:40 -06:00
Muhammad Atif Ali ef26ad96a9 chore(dogfood): bump modules versions 2024-03-19 23:54:16 +03:00
Kyle Carberry 4ae1f40eee chore: add docs for adding e2e tests (#12677) 2024-03-19 18:25:05 +00:00
Ben Potter c92ceffac9 chore: fix changelog typos (#12663) 2024-03-19 16:04:27 +00:00
Steven Masley 00283d1f8b chore: helm golden file test log output on failure (#12661)
Debugging this is hard without the actual helm error
2024-03-19 16:01:54 +00:00
Bruno Quaresma 23e3e4ce58 chore(site): upgrade msw to 2.0 (#12597)
Closes https://github.com/coder/coder/issues/11426
2024-03-19 09:30:20 -03:00
Danny Kopping 9cfd5baa91 feat(coderd): export metric indicating each experiment's status (#12657) 2024-03-19 14:11:27 +02:00
Ben Potter 1a9f7e7b00 hotfix(docs): we do not offer phone support yet (#12658) 2024-03-19 10:22:32 +00:00
Danny Kopping ab95ae827d feat(coderd): add enabled experiments to telemetry (#12656) 2024-03-19 11:05:29 +02:00
Steven Masley f0f9569d51 chore: enforce that provisioners can only acquire jobs in their own organization (#12600)
* chore: add org ID as optional param to AcquireJob
* chore: plumb through organization id to provisioner daemons
* add org id to provisioner domain key
* enforce org id argument
* dbgen provisioner jobs defaults to default org
2024-03-18 12:48:13 -05:00
Marcin Tojek 0e8ebb9b22 fix: fix flaky TestWorkspaceProxy_Server_PrometheusEnabled (#12645) 2024-03-18 16:43:41 +01:00
dependabot[bot] 2cd5fbc712 chore: bump github.com/ammario/tlru from 0.3.0 to 0.4.0 (#12635)
Bumps [github.com/ammario/tlru](https://github.com/ammario/tlru) from 0.3.0 to 0.4.0.
- [Commits](https://github.com/ammario/tlru/compare/v0.3.0...v0.4.0)

---
updated-dependencies:
- dependency-name: github.com/ammario/tlru
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 11:25:52 -04:00
dependabot[bot] 124da2e51c chore: bump github.com/u-root/u-root from 0.13.0 to 0.14.0 (#12636)
Bumps [github.com/u-root/u-root](https://github.com/u-root/u-root) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/u-root/u-root/releases)
- [Changelog](https://github.com/u-root/u-root/blob/main/RELEASES)
- [Commits](https://github.com/u-root/u-root/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: github.com/u-root/u-root
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 11:25:35 -04:00
Marcin Tojek cae769eac0 fix: implicit schema in dump (#12646) 2024-03-18 16:25:08 +01:00
dependabot[bot] 543a8ccb31 chore: bump github.com/coder/serpent (#12637)
Bumps [github.com/coder/serpent](https://github.com/coder/serpent) from 0.4.1-0.20240315163851-a0148c87ea3f to 0.5.0.
- [Commits](https://github.com/coder/serpent/commits/v0.5.0)

---
updated-dependencies:
- dependency-name: github.com/coder/serpent
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 09:39:28 -05:00
Muhammad Atif Ali f0c5e8e960 chore: reduce slack spam for manually run docs link checker workflows (#12630) 2024-03-18 17:23:54 +03:00
Marcin Tojek 15845d1a65 chore: use sqlc-vet to verify schema (#12642) 2024-03-18 15:23:25 +01:00
Ammar Bandukwala e5cc17af92 chore(cli): hide --organization (#12626)
It's potentially confusing to users since we aren't fleshing out organizations right now.
2024-03-18 09:09:26 -05:00
Dean Sheather cf50461ab4 fix: prevent single replica proxies from staying unhealthy (#12641)
In the peer healthcheck code, when an error pinging peers is detected we
write a "replicaErr" string with the error reason. However, if there are
no peer replicas to ping we returned early without setting the string to
empty. This would cause replicas that had peers (which were failing) and
then the peers left to permanently show an error until a new peer
appeared.

Also demotes DERP replica checking to a "warning" rather than an "error"
which should prevent the primary from removing the proxy from the region
map if DERP meshing is non-functional. This can happen without causing
problems if the peer is shutting down so we don't want to disrupt
everything if there isn't an issue.
2024-03-18 23:45:25 +10:00
Kyle Carberry 8a2f38a746 chore: simplify readme (#12639)
* chore: simplify readme

Closes #12628.

* Add link to module registry
2024-03-18 09:40:47 -04:00
Bruno Quaresma c84d96b747 fix(site): display not found page when pagination page is invalid (#12611) 2024-03-18 10:35:59 -03:00
dependabot[bot] b121f407f5 chore: bump golang.org/x/tools from 0.18.0 to 0.19.0 (#12632)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 14:44:16 +03:00
dependabot[bot] eb20d8cf18 chore: bump golang.org/x/mod from 0.15.0 to 0.16.0 (#12633)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 14:44:05 +03:00
dependabot[bot] fffa3dc422 chore: bump github.com/gohugoio/hugo from 0.123.3 to 0.124.0 (#12634)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 14:43:58 +03:00
Danny Kopping 93933d7905 feat(cli): show queue position during workspace builds (#12606) 2024-03-18 12:05:05 +02:00
dependabot[bot] c7597fdf02 chore: bump go.nhat.io/otelsql from 0.12.0 to 0.13.0 (#12521)
Bumps [go.nhat.io/otelsql](https://github.com/nhatthm/otelsql) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/nhatthm/otelsql/releases)
- [Commits](https://github.com/nhatthm/otelsql/compare/v0.12.0...v0.13.0)

---
updated-dependencies:
- dependency-name: go.nhat.io/otelsql
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 11:03:45 +03:00
dependabot[bot] 77cc170f04 chore: bump google.golang.org/api from 0.152.0 to 0.170.0 (#12607)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.152.0 to 0.170.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.152.0...v0.170.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 10:45:52 +03:00
Danny Kopping 53b58ed74b fix: correct troubleshooting links (#12608) 2024-03-18 08:52:20 +02:00
elasticspoon 5011edc292 fix(cli): show error/hide help for unsupported subcommands (#10760) (#12624)
Fixes #10760 

The coder CLI quietly accepts any subcommand arguments and silently swallows them.

Currently:
```sh
❯ coder | head -n5
coder v2.3.3+e491217

USAGE:
  coder [global-flags] <subcommand>
```

```sh
❯ coder idontexist | head -n5
coder v2.3.3+e491217

USAGE:
  coder [global-flags] <subcommand>
```

Now help output will not be show when there is an unknown subcommand error. Instead users will be given the command for the help output.

```sh
❯ coder idontexist
Encountered an error running "coder", see "coder --help" for more information 
error: unrecognized subcommand "idontexist"
```
```sh
❯ coder iexistbut idontexist
Encountered an error running "coder iexistbut", see "coder iexistbut --help" for more information 
error: unrecognized subcommand "idontexist"
```

Also this stuff: `Encountered an error running "coder iexistbut"... ` gets written to `os.Stdout` in `prettyErrorFormatter{w: os.Stderr, verbose: r.verbose}`, not sure how to test that output.
2024-03-17 22:17:43 -05:00
Steven Masley c189cc93e4 chore: bump gopkg.in/DataDog/dd-trace-go.v1 from 1.57.0 to 1.61.0 (#12610)
* chore: bump gopkg.in/DataDog/dd-trace-go.v1 from 1.57.0 to 1.61.0

* Fix tracer implementation
* Use alias vs 2 structs
2024-03-17 22:08:32 -05:00
Ammar Bandukwala b4c0fa80d8 chore(cli): rename Cmd to Command (#12616)
I think Command is cleaner and my original decision to use "Cmd"
a mistake.

Plus this creates better parity with cobra.
2024-03-17 09:45:26 -05:00
Muhammad Atif Ali 2a77580ba6 chore: fix false positives for docs links checker (#12623) 2024-03-17 16:44:45 +03:00
dependabot[bot] aa3ab209f3 ci: bump the github-actions group with 3 updates (#12622)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-17 13:27:15 +00:00
Gary Wang 1a5c5d0d57 fix: correct typo error about minTerraformVersion (#12621) 2024-03-17 13:18:46 +00:00
dependabot[bot] 4bdb019001 chore: bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#12614)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-17 16:12:51 +03:00
Gábor 9c69672382 fix(migration): removed hardcoded public (#12620) 2024-03-16 10:11:14 -04:00
Cian Johnston 9ff0bafcee fix(support): also sanitize agent environment (#12615) 2024-03-15 20:19:35 +00:00
Bruno Quaresma 6f0ba5bfe7 chore(site): add AgentLogs storybook (#12601) 2024-03-15 14:57:35 -03:00
Cian Johnston 25b605f764 fix(examples/lima/coder.yaml): update base image, remove usage of deprecated LIMA_CIDATA (#12613)
* fix(examples/lima/coder.yaml): update base image, remove usage of deprecated LIMA_CIDATA, name template consistently

* make fmt
2024-03-15 17:22:44 +00:00
Ammar Bandukwala 496232446d chore(cli): replace clibase with external coder/serpent (#12252) 2024-03-15 11:24:38 -05:00
Marcin Tojek bed2545636 docs: describe reference architectures (#12609) 2024-03-15 17:01:45 +01:00
Cian Johnston b0c4e7504c feat(support): add client magicsock and agent prometheus metrics to support bundle (#12604)
* feat(codersdk): add ability to fetch prometheus metrics directly from agent
* feat(support): add client magicsock and agent prometheus metrics to support bundle
* refactor(support): simplify AgentInfo control flow

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2024-03-15 15:33:49 +00:00
Eric Paulsen 4d9e6c0134 add updated architecture diagrams (#12584) 2024-03-15 11:09:03 -04:00
Cian Johnston 2fc9f097ed chore: apply linter auto-fixes (#12605) 2024-03-15 14:39:25 +00:00
Cian Johnston 18c1e02bf0 dogfood: replace siegfried with greenhill (#12599) 2024-03-15 23:33:43 +10:00
dependabot[bot] e1685b96e4 chore: bump golang.org/x/tools from 0.18.0 to 0.19.0 (#12519)
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.18.0 to 0.19.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 13:18:35 +00:00
Kyle Carberry 895df54051 fix: separate signals for passive, active, and forced shutdown (#12358)
* fix: separate signals for passive, active, and forced shutdown

`SIGTERM`: Passive shutdown stopping provisioner daemons from accepting new
jobs but waiting for existing jobs to successfully complete.

`SIGINT` (old existing behavior): Notify provisioner daemons to cancel in-flight jobs, wait 5s for jobs to be exited, then force quit.

`SIGKILL`: Untouched from before, will force-quit.

* Revert dramatic signal changes

* Rename

* Fix shutdown behavior for provisioner daemons

* Add test for graceful shutdown
2024-03-15 13:16:36 +00:00
dependabot[bot] 2c947c1921 chore: bump golang.org/x/mod from 0.15.0 to 0.16.0 (#12520)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.15.0 to 0.16.0.
- [Commits](https://github.com/golang/mod/compare/v0.15.0...v0.16.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 13:09:50 +00:00
dependabot[bot] 78f26bf24a chore: bump github.com/u-root/u-root from 0.13.0 to 0.14.0 (#12397)
Bumps [github.com/u-root/u-root](https://github.com/u-root/u-root) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/u-root/u-root/releases)
- [Changelog](https://github.com/u-root/u-root/blob/main/RELEASES)
- [Commits](https://github.com/u-root/u-root/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: github.com/u-root/u-root
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 08:55:10 -04:00
dependabot[bot] cf7f95b418 chore: bump golang.org/x/oauth2 from 0.17.0 to 0.18.0 (#12528)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.17.0 to 0.18.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 08:54:32 -04:00
dependabot[bot] 351706b896 chore: bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#12524)
Bumps google.golang.org/protobuf from 1.32.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 08:54:26 -04:00
Cian Johnston 2abc1cd2b7 feat(support): fetch agent network info over tailnet (#12577)
Adds agent-related information to support bundle command.
2024-03-15 11:01:39 +00:00
Cian Johnston 653ddccd8e fix(agent): remove unused token debug handler (#12602) 2024-03-15 09:43:36 +00:00
dependabot[bot] 8d7819f6d6 chore: bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#12404)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-14 19:37:32 -05:00
Steven Masley 29c8cf20e0 fix: handle CLI default organization when none exists in <v2.9.0 coderd (#12594)
* chore: 'coder org set' help message was incorrect
* fix: handler coder cli against older versions of Coder
2024-03-14 15:11:29 -05:00
Bruno Quaresma f78b5c1cfe chore(site): refactor logs and add stories (#12553) 2024-03-14 14:49:37 -03:00
Garrett Delfosse 0723dd3abf fix: ensure agent token is from latest build in middleware (#12443) 2024-03-14 12:27:32 -04:00
Cian Johnston 63696d762f feat(codersdk): add debug handlers for logs, manifest, and token to agent (#12593)
* feat(codersdk): add debug handlers for logs, manifest, and token to agent

* add more logging

* use io.LimitReader instead of seeking
2024-03-14 15:36:12 +00:00
Cian Johnston 135381bb4e chore(cli): skip support bundle test on windows (#12596) 2024-03-14 15:25:09 +00:00
Mathias Fredriksson 5dd436c19b feat(examples): add linting to all examples (#12595)
Fixes #12588
2024-03-14 16:49:44 +02:00
Steven Masley 410a7d54ee chore: external auth flow opens new window, it does not need an href (#12586) 2024-03-14 09:17:50 -05:00
Steven Masley 4cba83b30f test: apptest was accidently choosing ports in use (#12580)
Apptest requires a port without a listening server to test failure
cases. This port was chosen and had a chance of actually being
provisioned. To prevent this accident, a port <1k is chosen,
since those will never be allocated.
2024-03-14 08:54:12 -05:00
Danny Kopping 14130deb07 fix: clean template destination path for pull (#12559) 2024-03-14 12:41:23 +00:00
Cian Johnston 395bf54f4f chore(examples/templates/incus): fix metadata harder (#12589) 2024-03-14 11:43:34 +00:00
Muhammad Atif Ali 04b711f187 chore(examples/templates/incus): fix metadata (#12587) 2024-03-14 11:14:37 +00:00
Cian Johnston 3b406878e0 feat(agent): expose HTTP debug server over tailnet API (#12582) 2024-03-14 10:02:01 +00:00
Asher 0d16df9df9 fix: importing api into vscode-coder (#12570) 2024-03-13 10:05:44 -08:00
Kayla Washburn-Love efba477b36 fix: hide actions and notifications from deleted workspaces (#12563) 2024-03-13 11:43:48 -06:00
Bruno Quaresma 489b0ec497 chore(site): refactor useAuth and related hooks (#12567)
Close https://github.com/coder/coder/issues/12487
2024-03-13 13:24:18 -03:00
Spike Curtis fe6def31eb feat: upgrade tailscale fork to set TCP options for perf (#12574)
Upgrades our tailscale fork to include some TCP options tweaks for performance. c.f. https://github.com/coder/tailscale/pull/46
2024-03-13 16:02:02 +04:00
Michael Brewer 903f8b21c4 feat(site): add cpp icon (#12572)
* feat(site): add cpp icon

Add the C++ icon

Source is WikiCommons: https://en.m.wikipedia.org/wiki/File:ISO_C%2B%2B_Logo.svg

* fix: correct order
2024-03-13 07:49:03 -04:00
Danny Kopping da54c8a51f fix: fix data race in TestLabelsAggregation tests (#12578) 2024-03-13 13:47:22 +02:00
Danny Kopping 7a7105ad66 feat: make agent stats' cardinality configurable (#12535) 2024-03-13 12:03:36 +02:00
Bruno Quaresma e45d511f28 chore(site): add missing stories for templates (#12537) 2024-03-12 17:54:37 -03:00
Kayla Washburn-Love 301c60d824 chore(dogfood): add fish and helix ppa packages to dogfood (#12568) 2024-03-12 13:58:40 -06:00
Cian Johnston 096d472de9 chore(dogfood): update remaining focal PPAs to jammy (#12565) 2024-03-12 19:24:58 +00:00
Cian Johnston 901668ad4b feat(dogfood): add git from git-core ppa (#12564) 2024-03-12 17:28:07 +00:00
Bruno Quaresma 8489b4dfb1 chore(site): add WorkspaceBuildData stories (#12550) 2024-03-12 14:18:31 -03:00
Bruno Quaresma e947e0e829 chore(site): refactor dormant badge and add stories (#12555) 2024-03-12 14:17:55 -03:00
Ben Potter 321546474b docs: add v2.9.0 changelog (#12562)
* docs: add v2.9.0 changelog

* added sharable ports screenshot

* moved autostop visibility improvements from backend to dashboard, added screenshot

* move experiments to bottom

* added activity bump screenshot

---------

Co-authored-by: Stephen Kirby <me@skirby.dev>
2024-03-12 12:12:35 -05:00
Cian Johnston 47cb584052 fix(support): sanitize agent env (#12554) 2024-03-12 15:23:11 +00:00
Steven Masley 597694fbdd chore: bump migration file (#12556) 2024-03-12 14:55:45 +00:00
Steven Masley e11d3ca0ee chore: move default everyone group to a migration (#12435) 2024-03-12 09:27:36 -05:00
Bruno Quaresma f3083226ab chore: add package manager (#12551)
Every time I run `pnpm` in the project it adds the package manager attribute on package.json so I just decided to push it since it does not look like an issue and we can make sure everyone is running the same pnpm version.
2024-03-12 10:44:23 -03:00
Cian Johnston 7b081c873e fix(site): warn when user leaves template editor with un-built changes (#12548) 2024-03-12 13:08:54 +00:00
Danny Kopping 90d00190ea chore: remove pr_number param from deploy-pr.sh (#12549)
Field was removed in https://github.com/coder/coder/pull/11259

Signed-off-by: Danny Kopping <danny@coder.com>
2024-03-12 12:01:31 +00:00
Cian Johnston edc465c449 fix(site): TemplateVersionEditor: allow triggering builds on non-dirtied template version (#12547) 2024-03-12 11:35:16 +00:00
Spike Curtis 51707446d0 fix: stop holding Pubsub mutex while calling pq.Listener (#12518)
fixes #11950

https://github.com/coder/coder/issues/11950#issuecomment-1987756088 explains the bug

We were also calling into `Unlisten()` and `Close()` while holding the mutex.  I don't believe that `Close()` depends on the notification loop being unblocked, but it's hard to be sure, and the safest thing to do is assume it could block.

So, I added a unit test that fakes out `pq.Listener` and sends a bunch of notifies every time we call into it to hopefully prevent regression where we hold the mutex while calling into these functions.

It also removes the use of a `context.Context` to stop the PubSub -- it must be explicitly `Closed()`.  This simplifies a bunch of the logic, and is how we use the pubsub anyway.
2024-03-12 09:44:12 +04:00
dependabot[bot] 6f00ccfa64 chore: bump storj.io/drpc from 0.0.33-0.20230420154621-9716137f6037 to 0.0.33 (#12526)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-12 03:43:01 +00:00
dependabot[bot] da146e9655 chore: bump golang.org/x/crypto from 0.20.0 to 0.21.0 (#12523)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-12 03:41:54 +00:00
dependabot[bot] 242e4c4c85 chore: bump golang.org/x/term from 0.17.0 to 0.18.0 (#12525)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-12 03:40:00 +00:00
Michael Brewer f6ed81bc3b feat(site): add microsoft teams icon (#12513)
* feat(site): add microsoft teams icon

Attribution below:
Vectors and icons by <a href="https://github.com/garudatechnologydevelopers/sketch-icons?ref=svgrepo.com" target="_blank">Garuda Technology</a> in MIT License via <a href="https://www.svgrepo.com/" target="_blank">SVG Repo</a>


closes #12496

* fix(site): correct order of icons
2024-03-12 03:18:10 +00:00
Cian Johnston d704ff4570 chore(codersdk): explain format of codersdk.UpdateWorkspaceAutostartRequest.Schedule (#12539) 2024-03-11 22:50:38 +00:00
Kayla Washburn-Love a546cb8b32 chore: add stories to Search (#12457) 2024-03-11 12:16:31 -06:00
Bruno Quaresma 83af8674e8 chore(site): add CreateTokenPage story (#12472) 2024-03-11 14:10:19 -03:00
Steven Masley e3051dff0c chore: add workspace id filter on api (#12483)
* chore: add workspace id filter on api
2024-03-11 11:37:15 -05:00
Cian Johnston 8f40ee3465 Revert "feat: make agent stats' cardinality configurable (#12468)" (#12533)
This reverts commit 21d1873d97.
2024-03-11 14:33:36 +00:00
elasticspoon 773862a9f5 feat(cli): make url optional for login command (#10925) (#12466)
Allow `coder login` to log into existing deployment if available.

Update help and error messages to indicate that `coder login` is
available as a command.

Fixes #10925
Fixes #9551
2024-03-11 16:14:19 +02:00
Cian Johnston bed61f7d2a fix(coderd): correctly handle tar dir entries with missing path separator (#12479)
* coderd: add test to reproduce trailing directory issue
* coderd: add trailing path separator to dir entries when converting to zip
* provisionersdk: add trailing path separator to directory entries
2024-03-11 14:06:41 +00:00
Danny Kopping 21d1873d97 feat: make agent stats' cardinality configurable (#12468)
Closes #12221
2024-03-11 16:04:08 +02:00
Cian Johnston 0647ec1960 fix(coderd): prevent nil err deref (#12475) 2024-03-11 14:03:58 +00:00
Garrett Delfosse dc69341583 fix: make public menu item selectable (#12484) 2024-03-11 10:00:40 -04:00
Alessandro Varesi 5e9bf31229 fix: devcontainer-docker bad default directory (#12453) 2024-03-11 16:56:41 +03:00
Michael Brewer cef632b1fb feat(site): add dotnet icon (#12512) 2024-03-11 16:54:34 +03:00
Bruno Quaresma cd64e981b4 chore(site): add stories to 404 page (#12470)
Related to https://github.com/coder/coder/issues/12263
2024-03-11 10:36:06 -03:00
Cian Johnston b1ecc53033 chore(coderd): improve tests for tar<->zip conversion (#12477)
* improve tests for tar<->zip conversion
* set mode and modtime correctly when converting from zip to tar (#12476)
2024-03-11 13:29:57 +00:00
Bruno Quaresma 0220c97ef9 chore(site): add TableToolbar stories (#12473)
Related to https://github.com/coder/coder/issues/12263
2024-03-11 10:21:47 -03:00
Bruno Quaresma b8dd6b3aa2 chore(site): add Form storybook (#12469)
Related to #12260
2024-03-11 10:21:18 -03:00
Cian Johnston 1f276a22b3 chore(dogfood): update keys (#12515) 2024-03-11 13:07:48 +00:00
Mathias Fredriksson bae0a747ed test(coderd): skip flaky dau test (#12517)
* test(coderd): skip flaky dau test

* chore(coderd/database/dbpurge): fix failing test (#12530)

---------

Co-authored-by: Cian Johnston <cian@coder.com>
2024-03-11 12:54:38 +00:00
Michael Brewer 5296611a3f feat(site): add confluence icon (#12500)
Attribution for this icon below, if needed

Vectors and icons by <a href="https://github.com/vscode-icons/vscode-icons?ref=svgrepo.com" target="_blank">Vscode Icons</a> in MIT License via <a href="https://www.svgrepo.com/" target="_blank">SVG Repo</a>
2024-03-11 11:34:02 +03:00
dependabot[bot] 2b4560cc4b chore: bump github.com/fergusstrange/embedded-postgres (#12400)
Bumps [github.com/fergusstrange/embedded-postgres](https://github.com/fergusstrange/embedded-postgres) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/fergusstrange/embedded-postgres/releases)
- [Commits](https://github.com/fergusstrange/embedded-postgres/compare/v1.25.0...v1.26.0)

---
updated-dependencies:
- dependency-name: github.com/fergusstrange/embedded-postgres
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-11 00:16:16 +03:00
dependabot[bot] 6588cee38a chore: bump github.com/go-jose/go-jose/v3 from 3.0.1 to 3.0.3 (#12460)
Bumps [github.com/go-jose/go-jose/v3](https://github.com/go-jose/go-jose) from 3.0.1 to 3.0.3.
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Changelog](https://github.com/go-jose/go-jose/blob/v3.0.3/CHANGELOG.md)
- [Commits](https://github.com/go-jose/go-jose/compare/v3.0.1...v3.0.3)

---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-11 00:15:52 +03:00
Michael Smith 4d42c07c72 chore(site): update and refactor all custom hook tests that rely on React Router (#12219)
* chore: rename useTab to useSearchParamsKey and add test

* chore: mark old renderHookWithAuth as deprecated (temp)

* fix: update imports for useResourcesNav

* refactor: change API for useSearchParamsKey

* chore: let user pass in their own URLSearchParams value

* refactor: clean up comments for clarity

* fix: update import

* wip: commit progress on useWorkspaceDuplication revamp

* chore: migrate duplication test to new helper

* refactor: update code for clarity

* refactor: reorder test cases for clarity

* refactor: split off hook helper into separate file

* refactor: remove reliance on internal React Router state property

* refactor: move variables around for more clarity

* refactor: more updates for clarity

* refactor: reorganize test cases for clarity

* refactor: clean up test cases for useWorkspaceDupe

* refactor: clean up test cases for useWorkspaceDupe
2024-03-08 18:31:01 -05:00
Kayla Washburn-Love cf4f56dc2f chore: add stories for MoreMenu (#12464) 2024-03-08 12:01:48 -07:00
Kayla Washburn-Love 8d8220bb07 chore: add stories for Loader (#12445) 2024-03-08 11:35:14 -07:00
Ben Potter 1e17782ff6 docs: simplify install docs (#11946)
* docs: simplify install docs

* changes from feedback

* fmt

* fixups from feedback
2024-03-08 15:15:59 +00:00
Danny Kopping 7a92154e67 Install pnpm before calling pnpm exec in make stages (#12471)
Signed-off-by: Danny Kopping <danny@coder.com>
2024-03-08 14:33:28 +02:00
Bruno Quaresma 5b2acbc5b7 chore(site): add FileUpload stories (#12456)
Related to https://github.com/coder/coder/issues/12260
2024-03-08 09:08:44 -03:00
Bruno Quaresma 18d1c17db1 chore(site): add storybook for BuildAvatar and BuildIcon (#12455)
Related to https://github.com/coder/coder/issues/12260
2024-03-08 08:06:56 -03:00
Bruno Quaresma 060033e4ef fix(site): fix terminal size when displaying alerts (#12444)
Before - The terminal size does not fit the available space so the bottom is hidden.

https://github.com/coder/coder/assets/3165839/d08470b9-9fc6-476c-a551-8a3e13fc25bf

After - The terminal adjusts when there are alert changes.

https://github.com/coder/coder/assets/3165839/8cc32bfb-056f-47cb-97f2-3bb18c5fe906

Unfortunately, I don't think there is a sane way to automate tests for this but open to suggestions.

Close https://github.com/coder/coder/issues/7914
2024-03-08 07:38:40 -03:00
Dean Sheather d2a5b31b2b feat: add derp mesh health checking in workspace proxies (#12222) 2024-03-08 16:31:40 +10:00
Colin Adler 6b0b87eb27 fix: add --block-direct-connections to wsproxies (#12182) 2024-03-07 23:45:59 -06:00
Colin Adler 66154f937e fix(coderd): pass block endpoints into servertailnet (#12149) 2024-03-08 05:29:54 +00:00
Garrett Delfosse d2a74cf547 fix: display tooltip when selection is disabled (#12439) 2024-03-07 10:43:25 -05:00
Dean Sheather 586586e9dd fix: do not set max deadline for workspaces on template update (#12446)
* fix: do not set max deadline for workspaces on template update

When templates are updated and schedule data is changed, we update all
running workspaces to have up-to-date scheduling information that sticks
to the new policy.

When updating the max_deadline for existing running workspaces, if the
max_deadline was before now()+2h we would set the max_deadline to
now()+2h.

Builds that don't/shouldn't have a max_deadline have it set to 0, which
is always before now()+2h, and thus would always have the max_deadline
updated.

* test: add unit test to excercise template schedule bug
---------

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
2024-03-07 09:42:50 -06:00
Cian Johnston 17caf58b5e feat(support): add template info to support bundle (#12451)
Adds workspace build parameters, template, template version, and zipped template source to support bundle.
2024-03-07 14:43:46 +00:00
Bruno Quaresma db02c72ac6 chore(site): add storybook for terminal page (#12441) 2024-03-07 14:17:38 +00:00
Spike Curtis b96f6b48a4 fix: ensure ssh cleanup happens on cmd error
I noticed in my logs that sometimes `coder ssh` doesn't gracefully disconnect from the coordinator.

The cause is the `closerStack` construct we use in that function.  It has two paths to start closing things down:

1. explicit `close()` which we do in `defer`
2. context cancellation, which happens if the cli function returns an error

sometimes the ssh remote command returns an error, and this triggers context cancellation of the `closerStack`.  That is fine in and of itself, but we still want the explicit `close()` to wait until everything is closed before returning, since that's where we do cleanup, including the graceful disconnect.  Prior to this fix the `close()` just immediately exits if another goroutine is closing the stack.  Here we add a wait until everything is done.
2024-03-07 17:26:49 +04:00
Cian Johnston c8aa99a5b8 feat(coderd/database/dbfake): allow specifying fileID in TemplateVersionBuilder (#12450) 2024-03-07 12:36:11 +00:00
dependabot[bot] e4326947c4 chore: bump github.com/go-chi/httprate from 0.8.0 to 0.9.0 (#12401)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-07 16:57:21 +05:00
dependabot[bot] 14b1400968 chore: bump github.com/go-playground/validator/v10 from 10.18.0 to 10.19.0 (#12396)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-07 16:57:12 +05:00
dependabot[bot] de35755bd2 chore: bump github.com/hashicorp/hcl/v2 from 2.17.0 to 2.20.0 (#12398)
Bumps [github.com/hashicorp/hcl/v2](https://github.com/hashicorp/hcl) from 2.17.0 to 2.20.0.
- [Release notes](https://github.com/hashicorp/hcl/releases)
- [Changelog](https://github.com/hashicorp/hcl/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/hcl/compare/v2.17.0...v2.20.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/hcl/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-07 10:46:51 +00:00
Dean Sheather bd752a6d8b chore: embed static files in debug builds (#12449) 2024-03-07 09:23:28 +00:00
Cian Johnston 3e6e1e6f10 ci: add jnb wsproxy + update dogfood template (#12440)
Co-authored-by: Dean Sheather <dean@deansheather.com>
2024-03-06 18:54:26 +00:00
Dean Sheather 662be56d72 chore: rename migrations to fix main (#12442) 2024-03-06 18:28:53 +00:00
Steven Masley b5f866c1cb chore: add organization_id column to provisioner daemons (#12356)
* chore: add organization_id column to provisioner daemons
* Update upsert to include organization id on set
2024-03-06 12:04:50 -06:00
Dean Sheather 46a2ff1061 feat: allow setting port share protocol (#12383)
Co-authored-by: Garrett Delfosse <garrett@coder.com>
2024-03-06 09:23:57 -05:00
Steven Masley 23ff807a27 chore: remove autocreate orgs on CreateUser (#12434)
New users must be explictly given an organization to join.
Organizations should not be auto created as a side effect of
creating a new user.
2024-03-06 07:29:28 -06:00
Dean Sheather 842799847a chore: fix trivy scanning (#12421) 2024-03-05 19:04:16 -06:00
Michael Smith a92853c72d fix: ensure auto-workspace creation waits until all parameters are ready (#12419)
* fix: ensure auto-workspace creation waits until all parameters are ready

* refactor: move creation blocking logic to main callback

* fix: let creation start if experimental feature is off
2024-03-05 18:42:50 -05:00
Kayla Washburn-Love 0fe109d517 chore: sort imports in our typescript code (#12417) 2024-03-05 16:31:22 -07:00
Steven Masley 17c486c5e6 chore: ensure default org always exists (#12412)
* chore: ensure default org always exists

First user just joins the org created by the migration
2024-03-05 14:06:35 -06:00
Bruno Quaresma bc30c9c013 feat(site): warn user if they leave the editor without publishing (#12406) 2024-03-05 16:55:23 -03:00
Garrett Delfosse 61bd341a36 chore: change max share level on existing port shares (#12411) 2024-03-05 13:47:01 -05:00
Cian Johnston 5106d9fc47 feat(support): fetch data concurrently (#12385)
Modifies pkg support to fetch data concurrently
2024-03-05 17:41:42 +00:00
Bruno Quaresma fb88fa8603 feat(site): display error messages on ws and access url health pages (#12430)
Close https://github.com/coder/coder/issues/12408
2024-03-05 13:27:57 -03:00
Cian Johnston 4343998c37 chore(coderd): add tests for big oidc tokens (#12424)
- Adds two test cases for a 64k+ ID token and a 64k+ userinfo payload.
- Reformats the entire test cases array as instructed by CI
2024-03-05 14:46:00 +00:00
Marcin Tojek b1f9a6dc31 fix: use timestamptz instead of timestamp (#12425)
* fix: use timestampz instead of timestamp

* fix: timestamptz
2024-03-05 14:16:29 +00:00
Marcin Tojek 3e99c0373f fix: improve pagination parser (#12422) 2024-03-05 14:05:15 +00:00
Cian Johnston 61db293b33 feat(scripts/develop.sh): add --debug flag to develop.sh (#12423)
Adds a `--debug` flag to `scripts/develop.sh` that will start coder under `dlv debug` instead.
You can then use e.g. the following launch snippet to connect dlv:
```
    {
      "name": "Delve Remote",
      "type": "go",
      "request": "attach",
      "mode": "remote",
      "port": 12345,
    }
```

You can also run invididual CLI commands under dlv e.g.

```
debug=1 scripts/coder-dev.sh list
```

Also sets CGO_ENABLED=0 in develop.sh by default.
2024-03-05 13:29:08 +00:00
dependabot[bot] 8585863d0e chore: bump golang.org/x/crypto from 0.19.0 to 0.20.0 (#12403)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.19.0 to 0.20.0.
- [Commits](https://github.com/golang/crypto/compare/v0.19.0...v0.20.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-05 15:49:17 +05:00
Marcin Tojek e4fa212164 fix: always return count of workspaces (#12407) 2024-03-05 09:24:43 +01:00
Dean Sheather 0016b0200b chore: add test for workspace proxy derp meshing (#12220)
- Reworks the proxy registration loop into a struct (so I can add a `RegisterNow` method)
- Changes the proxy registration loop interval to 15s (previously 30s)
- Adds test which tests bidirectional DERP meshing on all possible paths between 6 workspace proxy replicas

Related to https://github.com/coder/customers/issues/438
2024-03-04 23:40:15 -08:00
Steven Masley 5c6974e55f feat: implement provisioner auth middleware and proper org params (#12330)
* feat: provisioner auth in mw to allow ExtractOrg

Step to enable org scoped provisioner daemons

* chore: handle default org handling for provisioner daemons
2024-03-04 15:15:41 -06:00
Colin Adler 926fd7ffa6 chore: add v2.6.1, v2.7.3, v2.8.4 release notes (#12415) 2024-03-04 12:23:01 -06:00
Alex 320c2eac6f Entra External Auth for ADO (#12201) 2024-03-04 12:12:46 -06:00
Colin Adler 4439a920e4 Merge pull request from GHSA-7cc2-r658-7xpf
This fixes a vulnerability with the `CODER_OIDC_EMAIL_DOMAIN` option,
where users with a superset of the allowed email domain would be allowed
to login. For example, given `CODER_OIDC_EMAIL_DOMAIN=google.com`, a
user would be permitted entry if their email domain was
`colin-google.com`.
2024-03-04 12:52:03 -05:00
Garrett Delfosse 8f190b2016 fix: disallow out of range ports (#12414) 2024-03-04 12:25:06 -05:00
Kayla Washburn-Love 3a86ae569a refactor: use TableEmpty in user settings (#12389) 2024-03-04 09:45:40 -07:00
Mathias Fredriksson 4ce1448bbe fix(cli): generate correctly named file in DumpHandler (#12409) 2024-03-04 18:35:33 +02:00
Bruno Quaresma afcea74462 fix(site): retry and debug passing build parameters options (#12384) 2024-03-04 10:25:53 -03:00
Kayla Washburn-Love af4d0b148b chore: add stories for Popover (#12387) 2024-03-01 15:43:35 -07:00
Michael Brewer 722ff50e59 fix: add service banner to workspace page (#12381) 2024-03-01 10:53:03 -07:00
Kayla Washburn-Love 4f0b885c30 chore: add stories for UserAvatar (#12376) 2024-03-01 10:50:17 -07:00
Kayla Washburn-Love 7824bee25f chore: add stories for Stack (#12375) 2024-03-01 10:26:50 -07:00
Kayla Washburn-Love f4c888f33e chore: add stories for Latency component (#12374) 2024-03-01 10:26:38 -07:00
Kayla Washburn-Love f00935baa6 chore: add stories for TableEmpty and TableLoader (#12373) 2024-03-01 10:26:30 -07:00
Cian Johnston b1c2fea78b feat(cli): add support cmd (#12328)
Part of #12163

- Adds a command coder support bundle <workspace> that generates a 
  support bundle and writes it to coder-support-$(date +%s).zip.
- Note: this is hidden currently until the rest of the functionality is fleshed out.
2024-03-01 17:13:50 +00:00
Colin Adler e5d911462f fix(tailnet): enforce valid agent and client addresses (#12197)
This adds the ability for `TunnelAuth` to also authorize incoming wireguard node IPs, preventing agents from reporting anything other than their static IP generated from the agent ID.
2024-03-01 09:02:33 -06:00
Colin Adler 7fbca62e08 chore: fix Test_parseInsightsStartAndEndTime flake (#12377)
Fixes https://github.com/coder/coder/issues/10600
2024-02-29 18:20:25 -06:00
Stephen Kirby 5a53afda46 minor change to quiet hours docs (#12338) 2024-02-29 14:50:00 -06:00
Bruno Quaresma 26b483d95e fix(site): fix form layout for tablet viewports (#12369) 2024-02-29 16:24:06 -03:00
Steven Masley 4006974a98 fix: external auth device flow, check both queries for errors (#12367)
* fix: external auth device flow, check both queries for errors
* Minor style update

---------

Co-authored-by: BrunoQuaresma <bruno_nonato_quaresma@hotmail.com>
2024-02-29 13:00:16 -06:00
Cian Johnston 9f3591add8 chore(cli): use xerrors.Errorf instead of fmt.Errorf (#12368) 2024-02-29 18:58:48 +00:00
Steven Masley cbcf4ef2c4 chore: add faking 429 responses from fake idp (#12365)
Required to trigger error condition in fe.
See pull (#12367)
2024-02-29 09:45:53 -06:00
Cian Johnston eba8cd7c07 chore: consolidate various randomPort() implementations (#12362)
Consolidates our existing randomPort() implementations to package testutil
2024-02-29 12:51:44 +00:00
Cian Johnston 4f87ba46f9 chore: update provisioner tag documentation with suggestions from #12315 (#12347)
- Adds more testcases to TestAcquirer_MatchTags
- Adds functionality to generate a table from above test
- Update provisioner tag documentation with generated table
- Apply other feedback from #12315
2024-02-29 12:31:11 +00:00
Cian Johnston e57c101200 feat: add support package and accompanying tests (#12289) 2024-02-29 11:58:33 +00:00
Cian Johnston 2bf3c72948 chore: add test for enterprise server cli (#12353) 2024-02-29 10:25:50 +00:00
Cian Johnston b17fcd9cff ci: use linter version from Dockerfile (#12354) 2024-02-29 09:53:32 +00:00
Kayla Washburn-Love b24ad1bbf0 refactor: show parameter suggestions from user history below field (#12340) 2024-02-28 15:29:48 -07:00
Kyle Carberry b2a5e2f4c0 fix: Increase license key rows (#12352)
It was pretty hard to tell when you pasted something in
this box with only displaying a single line.

This should help!
2024-02-28 21:57:10 +00:00
Steven Masley 97f083810f chore: provide usage instruction for CLI argument failures (#12309)
* chore: add usage to # cli arg failures
2024-02-28 12:10:17 -06:00
Marcin Tojek 30d9d84758 fix: use flag to enable Prometheus (#12345) 2024-02-28 17:58:03 +01:00
Dean Sheather bedd2c5922 fix: avoid race between replicas on start (#12344)
DERP mesh key setup would do a SELECT and then an INSERT on failure, without a lock. During some testing with multiple replicas, I managed to cause a replica to crash due to them initializing simultaneously.

Fixes:

Encountered an error running "coder server"
create coder API: insert mesh key: pq: duplicate key value violates unique constraint "site_configs_key_key"

Co-authored-by: Cian Johnston <cian@coder.com>
2024-02-28 16:14:11 +00:00
Bruno Quaresma 76273bf369 feat(site): display client errors in DERP Region health page (#12318) 2024-02-28 13:30:38 +00:00
Cian Johnston 1465ee2ed1 fix(coderd): use database.IsQueryCanceledError instead of xerrors.Is(err, context.Canceled) (#12325) 2024-02-28 21:19:57 +10:00
Marcin Tojek eb4a1e2568 feat: enable Prometheus endpoint for external provisioner (#12320) 2024-02-28 09:21:56 +01:00
Michael Smith 087f973415 refactor(site): clean up clipboard functionality and define tests (#12296)
* refactor: clean up and update API for useClipboard

* wip: commit current progress on useClipboard test

* docs: clean up wording on showCopySuccess

* chore: make sure tests can differentiate between HTTP/HTTPS

* chore: add test ID to dummy input

* wip: commit progress on useClipboard test

* wip: commit more test progress

* refactor: rewrite code for clarity

* chore: finish clipboard tests

* fix: prevent double-firing for button click aliases

* refactor: clean up test setup

* fix: rename incorrect test file

* refactor: update code to display user errors

* refactor: redesign useClipboard to be easier to test

* refactor: clean up GlobalSnackbar

* feat: add functionality for notifying user of errors (with tests)

* refactor: clean up test code

* refactor: centralize cleanup steps
2024-02-27 21:05:37 -05:00
dependabot[bot] e183843a16 chore: bump google.golang.org/grpc from 1.61.0 to 1.62.0 (#12301)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-28 02:37:44 +05:00
Bruno Quaresma 0fc1a9164e feat(site): support zip upload for template files (#12323)
Related to #11687
2024-02-27 18:00:23 -03:00
Eric Paulsen 0f9c142ea6 docs: add k8s security reference (#12334)
* docs: add k8s security reference

* make fmt
2024-02-27 15:59:11 -05:00
Kayla Washburn-Love 30772b80c1 Revert "ci: bump the github-actions group with 1 update (#12303)" (#12327)
This reverts commit 5757321ba2.
2024-02-27 19:49:50 +00:00
Kayla Washburn-Love b2413a593c chore: reimplement activity status and autostop improvements (#12175) 2024-02-27 11:06:26 -07:00
Spike Curtis 4e7beee102 feat: show tailnet peer diagnostics after coder ping (#12314)
Beginnings of a solution to #12297 

Doesn't cover disco or definitively display whether we successfully connected to DERP, but shows some checklist diagnostics for connecting to an agent.

For this first PR, I just added it to `coder ping` to see how we like it, but could be incorporated into `coder ssh` _et al._ after a timeout.

```
$ coder ping dogfood2
p2p connection established in 147ms
pong from dogfood2 p2p via  95.217.xxx.yyy:42631  in 147ms
pong from dogfood2 p2p via  95.217.xxx.yyy:42631  in 140ms
pong from dogfood2 p2p via  95.217.xxx.yyy:42631  in 140ms
✔ preferred DERP region 999 (Council Bluffs, Iowa)
✔ sent local data to Coder networking coodinator
✔ received remote agent data from Coder networking coordinator
    preferred DERP 10013 (Europe Fly.io (Paris))
    endpoints: 95.217.xxx.yyy:42631, 95.217.xxx.yyy:37576, 172.17.0.1:37576, 172.20.0.10:37576
✔ Wireguard handshake 11s ago
```
2024-02-27 22:04:46 +04:00
Mathias Fredriksson 32691e67e6 test(agent/agentscripts): fix test flake in TestEnv (#12326) 2024-02-27 17:58:10 +00:00
Kayla Washburn-Love cbaf1c65ef chore: clean out site/out/assets/ when building to prevent "too much data" errors (#12313) 2024-02-27 10:45:57 -07:00
Cian Johnston b9e2d0a400 fix(coderd): mark provisioner daemon psk as secret (#12322)
* fix(coderd): mark provisioner daemon psk as secret

Marks provisioner daemon PSK with the secret annotation.
This ensures it will be scrubbed from API requests to
/api/v2/deployment/config.

* make gen
2024-02-27 16:33:32 +00:00
Steven Masley 19baca55da feat: implement create org commands from cli (#12308)
* feat: implement create org commands from cli
2024-02-27 10:13:08 -06:00
dependabot[bot] 5757321ba2 ci: bump the github-actions group with 1 update (#12303)
Bumps the github-actions group with 1 update: [chromaui/action](https://github.com/chromaui/action).


Updates `chromaui/action` from 10 to 11
- [Release notes](https://github.com/chromaui/action/releases)
- [Commits](https://github.com/chromaui/action/compare/v10...v11)

---
updated-dependencies:
- dependency-name: chromaui/action
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-27 11:53:48 -03:00
Cian Johnston 1d65e36b89 ci: increase connection hard limit for fly.io wsproxies (#12319) 2024-02-27 13:57:24 +00:00
Cian Johnston 392fecee87 chore(docs): update external provisioners documentation (#12315) 2024-02-27 13:05:28 +00:00
Cian Johnston 96c9838ce3 fix(cli): scaletest: do not screenshot if verbose=false (#12317) 2024-02-27 12:35:48 +00:00
Bruno Quaresma 2ca8248315 chore(site): apply code conventions (#12316) 2024-02-27 12:24:07 +00:00
dependabot[bot] 5a0d9db6c3 chore: bump github.com/elastic/go-sysinfo from 1.12.0 to 1.13.1 (#12213)
Bumps [github.com/elastic/go-sysinfo](https://github.com/elastic/go-sysinfo) from 1.12.0 to 1.13.1.
- [Release notes](https://github.com/elastic/go-sysinfo/releases)
- [Commits](https://github.com/elastic/go-sysinfo/compare/v1.12.0...v1.13.1)

---
updated-dependencies:
- dependency-name: github.com/elastic/go-sysinfo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-27 15:19:43 +05:00
Selina NN 431bf5cf3b Update artifactory-integration.md (#12311)
Update date year
2024-02-27 14:57:26 +05:00
Asher f74532ff50 feat: audit oauth2 app management (#12275)
* Audit oauth2 app management
* Use 201 for creating secrets
2024-02-26 23:52:08 +00:00
Steven Masley 6b866b3f48 feat: set sane default for gitea external auth (#12306)
* feat: external auth defaults for gitea

Add some sane defaults for gitea to make it easier to configure
2024-02-26 12:35:18 -06:00
Steven Masley 70ccefc357 feat: set organization context in coder organizations (#12265)
* feat: add coder organizations set to change org context

`coder organizations set <org>`
2024-02-26 11:39:26 -06:00
Steven Masley 748cf4b2c4 feat: implement global flag for org selection (#12276)
* feat: implement global flag for org selection

Any command can use '-z' to override org context
2024-02-26 11:38:49 -06:00
dependabot[bot] 5a41385400 chore: bump github.com/gohugoio/hugo from 0.122.0 to 0.123.3 (#12302)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-26 21:18:19 +05:00
Steven Masley d2998c6b7b feat: implement organization context in the cli (#12259)
* feat: implement organization context in the cli

`coder org show current`
2024-02-26 10:03:49 -06:00
Steven Masley f44c89d200 chore: enforce orgid in audit logs where required (#12283)
* chore: enforce orgid in audit logs where required
2024-02-26 08:27:33 -06:00
Cian Johnston 74b749b890 chore(coderd): add test to assert agent token invalid when workspace deleted (#12290) 2024-02-26 13:27:00 +00:00
Muhammad Atif Ali 7eed40bd99 chore(dogfood): bump jetbrains-gateway module to 1.0.6 (#12298) 2024-02-26 11:57:40 +00:00
Spike Curtis b0afffbafb feat: use v2 API for agent metadata updates (#12281)
Switches the agent to report metadata over the v2 API.

Fixes #10534
2024-02-26 09:50:19 +04:00
Gary Reynolds 7a245e61b1 chore(docs): inline OIDC flow diagram (#12255)
When viewing the Authentication page, the diagram showing the flow is a useful
resource for understanding the rest of the page.

Rather than linking to a specific version of the SVG, inline it as part of the
documentation.
2024-02-26 04:48:01 +00:00
Michael Brewer 245e280531 docs: add gitlab self-managed example (#12295) 2024-02-25 10:11:13 -05:00
Eric Paulsen fb198ac99c docs: add steps for postgres server verification (#12072)
* docs: add steps for postgres server verification

* make: fmt

* refactor to guide

* add manifest
2024-02-25 01:16:56 +00:00
dependabot[bot] 7e797e90ac chore: bump golang.org/x/tools from 0.17.0 to 0.18.0 (#12209)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-24 10:42:50 +05:00
Steven Masley c33c452663 fix: use default org over index [0] for new scim (#12284) 2024-02-23 15:31:36 -06:00
Colin Adler 0dd126e025 chore: upgrade github.com/hashicorp/hc-install to v0.6.3 (#12288)
Removes a dependency on a vulnerable version of github.com/cloudflare/circl
2024-02-23 13:41:58 -06:00
Bruno Quaresma 79480ca587 feat(site): display build logs on template creation (#12271) 2024-02-23 12:23:52 -07:00
Steven Masley 13359aa16f chore: drop github per user rate limit tracking (#12286)
* chore: drop github per user rate limit tracking

Rate limits for authenticated requests are per user.
This would be an excessive number of prometheus labels,
so we only track the unauthorized limit.
2024-02-23 11:17:52 -06:00
Marcin Tojek 90db6683c4 fix: refresh entitlements after creating first user (#12285) 2024-02-23 16:48:24 +00:00
Cian Johnston 2cb9bfd517 refactor(coderd): move healthcheck report structs to codersdk (#12279)
Moves healthcheck report-related structs from coderd/healthcheck to codersdk
This prevents an import cycle when adding a codersdk.Client method to hit /api/v2/debug/health.
2024-02-23 13:13:28 +00:00
Spike Curtis aa7a9f5cc4 feat: use v2 API for agent lifecycle updates (#12278)
Agent uses the v2 API to post lifecycle updates.

Part of #10534
2024-02-23 15:24:28 +04:00
Dean Sheather ee7828a166 chore: fix wsproxy test flake (#12280)
* chore: fix wsproxy test flake

* fixup! chore: fix wsproxy test flake
2024-02-23 21:19:54 +10:00
Spike Curtis 4cc132cea0 feat: switch agent to use v2 API for sending logs (#12068)
Changes the agent to use the new v2 API for sending logs, via the logSender component.

We keep the PatchLogs function around, but deprecate it so that we can test the v1 endpoint.
2024-02-23 11:27:15 +04:00
Spike Curtis af3fdc68c3 chore: refactor agent routines that use the v2 API (#12223)
In anticipation of needing the `LogSender` to run on a context that doesn't get immediately canceled when you `Close()` the agent, I've undertaken a little refactor to manage the goroutines that get run against the Tailnet and Agent API connection.

This handles controlling two contexts, one that gets canceled right away at the start of graceful shutdown, and another that stays up to allow graceful shutdown to complete.
2024-02-23 11:04:23 +04:00
Kayla Washburn-Love 66585f042f feat: support markdown in update messages (#12273) 2024-02-22 16:14:06 -07:00
Kayla Washburn-Love 7e6cb66a50 feat(site): allow creating a workspace without connecting optional external auth providers (#12251) 2024-02-22 10:27:36 -07:00
Kayla Washburn-Love b8a53230c7 chore: revert "refactor(site): verify external auth before display ws form (#11777)" (#12183) 2024-02-22 09:44:30 -07:00
Cian Johnston 53e8f9c0f9 fix(coderd): only allow untagged provisioners to pick up untagged jobs (#12269)
Alternative solution to #6442

Modifies the behaviour of AcquireProvisionerJob and adds a special case for 'un-tagged' jobs such that they can only be picked up by 'un-tagged' provisioners.

Also adds comprehensive test coverage for AcquireJob given various combinations of tags.
2024-02-22 15:04:31 +00:00
Marcin Tojek aa7a12a5ec docs: document Terraform variables (#12270) 2024-02-22 15:26:53 +01:00
Steven Masley d4d8424ce0 fix: fix GetOrganizationsByUserID error when multiple organizations exist (#12257)
* test: fetching user orgs fails if multi orgs in pg db
* fix: GetOrganizationsByUserID fixed if multi orgs exist
2024-02-22 08:14:48 -06:00
Spike Curtis da376549a3 fix: stop waiting for Agent in a goroutine in ssh test (#12268)
Fixes race seen here: https://github.com/coder/coder/runs/21852483781

What happens is that the agent connects, completes the test, and then disconnects before the Eventually condition runs.  The waiter then times out because it's looking for a connected agent.

Then, since it's a `require` in a goroutine, that causes the `tGo` cleanup to hang and the whole test suite to timeout after 10 minutes.

Anyway, `agenttest.New` doesn't block, and we don't actually need to wait for the agent to connect, since a successful SSH session is evidence that it connected.
2024-02-22 17:01:06 +04:00
dependabot[bot] a31a05e2cb chore: bump github.com/valyala/fasthttp from 1.51.0 to 1.52.0 (#12210)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-22 11:59:29 +05:00
dependabot[bot] 307a206605 chore: bump github.com/prometheus/client_model from 0.5.0 to 0.6.0 (#12212)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-22 11:59:17 +05:00
Asher 51d178d538 feat: add OAuth2 user settings page (#12237) 2024-02-21 14:16:55 -09:00
dependabot[bot] 3cbe14fdad chore: bump ip from 2.0.0 to 2.0.1 in /site (#12238)
Bumps [ip](https://github.com/indutny/node-ip) from 2.0.0 to 2.0.1.
- [Commits](https://github.com/indutny/node-ip/compare/v2.0.0...v2.0.1)

---
updated-dependencies:
- dependency-name: ip
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 14:06:43 -09:00
Asher 7eb2beccea fix: redirect from oauth2 authorization page (#12241)
At the moment it just says "you are not authorized", but we want to
automatically redirect to the login page.
2024-02-21 13:30:33 -09:00
Steven Masley c3a7b13690 chore: remove organization requirement from convertGroup() (#12195)
* feat: convertGroups() no longer requires organization info

Removing role information from some users in the api. This info is
excessive and not required. It is costly to always include
2024-02-21 15:58:11 -06:00
Steven Masley 3f65bd14cc fix: ignore surronding whitespace for cli config (#12250)
* fix: ignore surronding whitespace for cli config

Cli config files break if you edit them manually with any editor.
Editors drop a newline at the end, and we not break on this.
If a developer manually edits a file, it should still work
2024-02-21 13:03:41 -06:00
Kayla Washburn-Love 475c3650ca feat: add support for optional external auth providers (#12021) 2024-02-21 11:18:38 -07:00
Bruno Quaresma 78c9f82719 fix(site): fix error when typing long number on ttl (#12249) 2024-02-21 17:29:29 +00:00
Michael Smith 1d254f4680 fix: add tests and improve accessibility for useClickable (#12218) 2024-02-21 10:59:13 -05:00
Bruno Quaresma a827185b6d refactor: move auto fill feature into an experiment (#12230) 2024-02-21 11:48:34 -03:00
Marcin Tojek c230bcf5ca fix: previous parameter value is not a number (#12246) 2024-02-21 15:44:45 +01:00
Bruno Quaresma b4fb754b2d feat(site): show previous agent scripts logs (#12233) 2024-02-21 11:42:34 -03:00
Bruno Quaresma 0398e3c531 chore(site): fix storybook for agent row with port forward button (#12247) 2024-02-21 14:32:39 +00:00
Bruno Quaresma cc4cefbbee chore(site): fix storybook test (#12245) 2024-02-21 14:17:18 +00:00
Bruno Quaresma ebe05820c9 fix(site): fix web terminal bottom overflow (#12228) 2024-02-21 11:02:53 -03:00
dependabot[bot] 91c3df785f chore: bump github.com/aws/smithy-go from 1.19.0 to 1.20.0 (#12206)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 13:09:18 +05:00
dependabot[bot] 519cf5935f chore: bump github.com/prometheus/common from 0.46.0 to 0.47.0 (#12207)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 11:36:52 +05:00
Asher 3d742f64e6 fix: move oauth2 routes (#12240)
* fix: move oauth2 routes

From /login/oauth2/* to /oauth2/*.

/login/oauth2 causes /login to no longer get served by the frontend,
even if nothing is actually served on /login itself.

* Add forgotten comment on delete
2024-02-20 17:01:25 -09:00
Asher 4d39da294e feat: add oauth2 token exchange (#12196)
Co-authored-by: Steven Masley <stevenmasley@gmail.com>
2024-02-20 14:58:43 -09:00
Steven Masley 07cccf9033 feat: disable directory listings for static files (#12229)
* feat: disable directory listings for static files

Static file server handles serving static asset files (js, css, etc).
The default file server would also list all files in a directory.
This has been changed to only serve files.
2024-02-20 15:50:30 -06:00
Steven Masley 2dac34276a fix: add postgres triggers to remove deleted users from user_links (#12117)
* chore: add database test fixture to insert non-unique linked_ids
* chore: create unit test to exercise failed email change bug
* fix: add postgres triggers to keep user_links clear of deleted users
* Add migrations to prevent deleted users with links
* Force soft delete of users, do not allow un-delete
2024-02-20 13:19:38 -06:00
Garrett Delfosse b342bd7869 feat: add port sharing frontend (#12119) 2024-02-20 13:26:34 -05:00
Bruno Quaresma 0021c2f906 fix(site): fix parameters field size (#12231) 2024-02-20 13:54:07 -03:00
Marcin Tojek 57bf997369 feat: support custom validation errors for number-typed parameters (#12224) 2024-02-20 16:32:03 +01:00
Michael Smith 6414b7aade chore(site): refactor tests for global hooks   (#12216)
* refactor: clean up tests for debounce

* refactor: clean up tests for useCustomEvent

* refactor: clean up events file

* refactor: clean up tests for hookPolyfills
2024-02-20 09:19:43 -05:00
Michael Smith d6ae9d8548 revert: remove anti-flicker clipboard styling (#12227)
- These CSS changes were for making sure there weren't layout shifts
  when using the non-secure clipboard fallback, which could cause janky
  UI flickers. It seems to be breaking things for some users on HTTP-only
  connections, though.
2024-02-20 14:14:25 +00:00
Cian Johnston 643c3ee54b refactor(provisionerd): move provisionersdk.VersionCurrent -> provisionerdproto.VersionCurrent (#12225) 2024-02-20 12:44:19 +00:00
Cian Johnston c62a8b0bee fix(helm)!: remove prometheus-http port declaration from coderd service spec (#12214)
This PR removes the prometheus-http port entirely from the coder service specification (originally added in #10448). It also removes the Helm value coder.service.prometheusNodePort.

Rationale: some cloud providers will helpfully expose all ports on a LoadBalancer service for you. The net effect of this is that setting CODER_PROMETHEUS_ENABLE will end up exposing port 2112 on your coderd service to the internet, which is likely undesired behaviour.
2024-02-20 11:36:17 +00:00
Mathias Fredriksson b1c0b39d88 feat(agent): add script data dir for binaries and files (#12205)
The agent is extended with a `--script-data-dir` flag, defaulting to the
OS temp dir. This dir is used for storing `coder-script-data/bin` and
`coder-script/[script uuid]`. The former is a place for all scripts to
place executable binaries that will be available by other scripts, SSH
sessions, etc. The latter is a place for the script to store files.

Since we default to OS temp dir, files are ephemeral by default. In the
future, we may consider adding new env vars or changing the default
storage location. Workspace startup speed could potentially benefit from
scripts being able to skip steps that require downloading software. We
may also extend this with more env variables (e.g. persistent storage in
HOME).

Fixes #11131
2024-02-20 13:26:18 +02:00
Spike Curtis ab4cb66e00 feat: add WaitUntilEmpty to LogSender (#12159)
We'll need this to be able to tell when all outstanding logs have been sent, as part of graceful shutdown.
2024-02-20 11:11:31 +04:00
Spike Curtis 081e37d7d9 chore: move LogSender to agentsdk (#12158)
Moves the LogSender to agentsdk and deprecates LogsSender based on the v1 API.
2024-02-20 10:44:20 +04:00
Dean Sheather 9861830e87 fix: never send local endpoints if disabled (#12138) 2024-02-20 15:51:25 +10:00
Mathias Fredriksson c63f569174 refactor(agent/agentssh): move envs to agent and add agentssh config struct (#12204)
This commit refactors where custom environment variables are set in the
workspace and decouples agent specific configs from the `agentssh.Server`.
To reproduce all functionality, `agentssh.Config` is introduced.

The custom environment variables are now configured in `agent/agent.go`
and the agent retains control of the final state. This will allow for
easier extension in the future and keep other modules decoupled.
2024-02-19 16:30:00 +02:00
Colin Adler 817cc78b94 fix(examples): remove dead code comment (#12194) 2024-02-17 17:38:19 +00:00
Mathias Fredriksson 0442ee5fa8 fix(agent/reconnectingpty): fix screen startup speed by disabling messages (#12190) 2024-02-16 22:37:02 +02:00
Cian Johnston a2cbb0f87f fix(enterprise/coderd): check provisionerd API version on connection (#12191) 2024-02-16 18:43:07 +00:00
Steven Masley f17149c59d feat: set groupsync to use default org (#12146)
* fix: assign new oauth users to default org

This is not a final solution, as we eventually want to be able
to map to different orgs. This makes it so multi-org does not break oauth/oidc.
2024-02-16 11:09:19 -06:00
Kayla Washburn-Love dbaafc863c chore: update no-restricted-imports lint rule (#12180)
- prevent importing from the "monolith" lodash module. individual modules are better for tree shaking.
- prevent importing `useTheme` and types from @mui/material/styles. prefer importing from @emotion/react.
2024-02-16 09:54:40 -07:00
Steven Masley 75870c22ab fix: assign new oauth users to default org (#12145)
* fix: assign new oauth users to default org

This is not a final solution, as we eventually want to be able
to map to different orgs. This makes it so multi-org does not break oauth/oidc.
2024-02-16 08:47:26 -06:00
Steven Masley 2a8004b1b2 feat: use default org for PostUser (#12143)
Instead of assuming only 1 org exists, this uses the
is_default org to place a user in if not specified.
2024-02-16 08:28:36 -06:00
Marcin Tojek 0e1bad4f82 docs: fix header font (#12193) 2024-02-16 13:32:45 +00:00
Muhammad Atif Ali 799d71f6b2 docs: simplify docker installation docs (#12187) 2024-02-16 12:53:03 +00:00
Bruno Quaresma be1edc3995 fix(site): fix language detection for Dockerfile (#12188) 2024-02-16 12:50:36 +00:00
Marcin Tojek 41647ca984 docs: describe resource ordering in UI (#12185) 2024-02-16 13:33:57 +01:00
Bruno Quaresma df297627c2 fix(site): match activity bump text with template settings (#12170)
Close https://github.com/coder/coder/issues/12130
2024-02-16 09:33:15 -03:00
Muhammad Atif Ali 99dbeb4a85 ci: fix broken dogfood workflow (#12186) 2024-02-16 14:14:35 +03:00
Muhammad Atif Ali 8ca2add6dc chore(dogfood): revert to pre-artifactory state (#12169) 2024-02-16 13:47:15 +03:00
Colin Adler 97e4d51953 fix(cli/clibase): don't error on required flags with --help (#12181) 2024-02-15 23:41:46 +00:00
Michael Smith fbd436cc2c fix: improve clipboard support on HTTP connections and older browsers (#12178)
* fix: add future-proofing for clipboard copies on http connections

* docs: clean up comment formatting
2024-02-15 16:44:53 -05:00
Colin Adler 8a9f59a4bb fix(cli): avoid panic when external auth name isn't provided (#12177)
Fixes https://github.com/coder/coder/issues/10216
2024-02-15 15:17:16 -06:00
Colin Adler 4c3d44658d fix(codersdk): correctly log coordination error (#12176) 2024-02-15 20:47:12 +00:00
Steven Masley 2bf2f88b09 feat: implement 'is_default' org field (#12142)
The first organization created is now marked as "default". This is
to allow "single org" behavior as we move to a multi org codebase.

It is intentional that the user cannot change the default org at this
stage. Only 1 default org can exist, and it is always the first org.

Closes: https://github.com/coder/coder/issues/11961
2024-02-15 11:01:16 -06:00
dependabot[bot] a67362fdb1 chore: bump github.com/u-root/u-root from 0.12.0 to 0.13.0 (#12100)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-15 19:25:52 +03:00
Marcin Tojek 5aa5ff1bde chore: deprecate API workspace build resources (#12167) 2024-02-15 17:13:44 +01:00
Spike Curtis 2aff014e5d feat: add logSender for sending logs on agent v2 API (#12046)
Adds a new subcomponent of the agent for queueing up logs until they can be sent over the Agent API.

Subsequent PR will change the agent to use this instead of the HTTP API for posting logs.

Relates to #10534
2024-02-15 16:57:17 +04:00
Spike Curtis 627232eae9 fix: fix pgcoord to delete coordinator row last (#12155)
Fixes #12141
Fixes #11750

PGCoord shutdown was uncoordinated, so an update at an inopportune time during shutdown would be rejected because the coordinator row was already deleted.

This PR ensures that the PGCoord subcomponents that write updates are shut down before we take down the heartbeats, which is responsible for deleting the coordinator row.
2024-02-15 16:34:29 +04:00
Marcin Tojek 7a453608c9 feat: support order property of coder_agent (#12121) 2024-02-15 13:33:13 +01:00
Sulochan c66e665864 docs: add kubevirt coder template in list of community templates (#12113) 2024-02-15 13:18:10 +03:00
Marcin Tojek 8cc62fb221 fix(site): ignore fileInfo if file is missing (#12154) 2024-02-15 09:15:22 +00:00
Muhammad Atif Ali d9f99da327 chore(docs): update artifactory-integration guide (#12153) 2024-02-15 11:20:50 +03:00
Spike Curtis 2d0b9106c0 fix: change servertailnet to register the DERP dialer before setting DERP map (#12137)
I noticed a possible race where tailnet.Conn can try to dial the embedded region before we've set our custom dialer that send the DERP in-memory.  This closes that race and adds a test case for servertailnet with no STUN and an embedded relay
2024-02-15 10:51:12 +04:00
dependabot[bot] 1bb4aecf49 chore: bump golang.org/x/oauth2 from 0.16.0 to 0.17.0 (#12099)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.16.0 to 0.17.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-14 13:37:30 -06:00
Cian Johnston d6b025db14 Revert "feat: add activity status and autostop reason to workspace overview (#11987)" (#12144)
Related to https://github.com/coder/coder/pull/11987

This reverts commit d37b131.
2024-02-14 17:14:49 +00:00
Spike Curtis 04991f425a fix: set node callback each time we reinit the coordinator in servertailnet (#12140)
I think this will resolve #12136 but lets get a proper test at the system level before closing.

Before this change, we only register the node callback at start of day for the server tailnet.  If the coordinator changes, like we know happens when we are licensed for the PGCoordinator, we close the connection to the old coord, and open a new one to the new coord.

The callback is designed to direct the updates to the new coordinator, but there is nothing that specifically triggers it to fire after we connect to the new coordinator.

If we have STUN, then period re-STUNs will generally get it to fire eventually, but without STUN it we could go indefinitely without a callback.

This PR changes the servertailnet to re-register the callback each time we reconnect to the coordinator.  Registering a callback (even if it's the same callback) triggers an immediate call with our node information, so the new coordinator will have it.
2024-02-14 20:45:31 +04:00
Spike Curtis 5a0d240bc3 feat: expose DERP server debug metrics (#12135)
Adds some debug endpoints for looking into the DERP server.

The `api/v2/debug/derp/traffic` endpoint requires the `ss` utility to be present in order to function.  I have *not* added the `iproute2` package to our base image as it adds 11MB, so this endpoint won't be useful by default.  However, in a debugging situation, we could exec into the container and then `apk add iproute2`, or build a special debug image.

The `api/v2/debug/expvar` handler contains DERP metrics as well as commandline and memstats.

Example:

```
{
"alert_failed": 0,
"alert_generated": 0,
"cmdline": ["/Users/spike/repos/coder/build/coder_darwin_arm64","--global-config","/Users/spike/repos/coder/.coderv2","server","--http-address","0.0.0.0:3000","--swagger-enable","--access-url","http://127.0.0.1:3000","--dangerous-allow-cors-requests=true"],
"derp": {"accepts": 1, "average_queue_duration_ms": 0, "bytes_received": 0, "bytes_sent": 0, "counter_packets_dropped_reason": {"gone_disconnected": 0, "gone_not_here": 0, "queue_head": 0, "queue_tail": 0, "unknown_dest": 0, "unknown_dest_on_fwd": 0, "write_error": 0}, "counter_packets_dropped_type": {"disco": 0, "other": 0}, "counter_packets_received_kind": {"disco": 0, "other": 0}, "counter_tcp_rtt": {}, "counter_total_dup_client_conns": 0, "gauge_clients_local": 1, "gauge_clients_remote": 0, "gauge_clients_total": 1, "gauge_current_connections": 1, "gauge_current_dup_client_conns": 0, "gauge_current_dup_client_keys": 0, "gauge_current_file_descriptors": 0, "gauge_current_home_connections": 1, "gauge_memstats_sys0": 20874504, "gauge_watchers": 0, "got_ping": 0, "home_moves_in": 0, "home_moves_out": 0, "multiforwarder_created": 0, "multiforwarder_deleted": 0, "packet_forwarder_delete_other_value": 0, "packets_dropped": 0, "packets_forwarded_in": 0, "packets_forwarded_out": 0, "packets_received": 0, "packets_sent": 0, "peer_gone_disconnected_frames": 0, "peer_gone_not_here_frames": 0, "sent_pong": 0, "unknown_frames": 0, "version": "1.47.0-dev20240214-t64db8c604"},
"memstats": {"Alloc":286506256,"TotalAlloc":297594632,"Sys":310621512,"Lookups":0,"Mallocs":304204,"Frees":171570,"HeapAlloc":286506256,"HeapSys":294060032,"HeapIdle":3694592,"HeapInuse":290365440,"HeapReleased":3620864,"HeapObjects":132634,"StackInuse":3735552,"StackSys":3735552,"MSpanInuse":347256,"MSpanSys":358512,"MCacheInuse":9600,"MCacheSys":15600,"BuckHashSys":1469877,"GCSys":9434896,"OtherSys":1547043,"NextGC":551867656,"LastGC":1707892877408883000,"PauseTotalNs":1247000,"PauseNs":[200333,229375,239875,209542,106958,203792,57125,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"PauseEnd":[1707892876217481000,1707892876219726000,1707892876222273000,1707892876226151000,1707892876234815000,1707892877398146000,1707892877408883000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"NumGC":7,"NumForcedGC":0,"GCCPUFraction":0.0022425810335762954,"EnableGC":true,"DebugGC":false,"BySize":[{"Size":0,"Mallocs":0,"Frees":0},{"Size":8,"Mallocs":14396,"Frees":9143},{"Size":16,"Mallocs":89090,"Frees":50507},{"Size":24,"Mallocs":40839,"Frees":24456},{"Size":32,"Mallocs":22404,"Frees":12379},{"Size":48,"Mallocs":51174,"Frees":23718},{"Size":64,"Mallocs":15406,"Frees":3501},{"Size":80,"Mallocs":6688,"Frees":2352},{"Size":96,"Mallocs":2567,"Frees":374},{"Size":112,"Mallocs":19371,"Frees":16883},{"Size":128,"Mallocs":2873,"Frees":1061},{"Size":144,"Mallocs":5600,"Frees":2742},{"Size":160,"Mallocs":2159,"Frees":622},{"Size":176,"Mallocs":454,"Frees":86},{"Size":192,"Mallocs":227,"Frees":128},{"Size":208,"Mallocs":1407,"Frees":732},{"Size":224,"Mallocs":1365,"Frees":1090},{"Size":240,"Mallocs":82,"Frees":48},{"Size":256,"Mallocs":310,"Frees":162},{"Size":288,"Mallocs":1945,"Frees":562},{"Size":320,"Mallocs":1200,"Frees":458},{"Size":352,"Mallocs":133,"Frees":33},{"Size":384,"Mallocs":582,"Frees":51},{"Size":416,"Mallocs":747,"Frees":200},{"Size":448,"Mallocs":113,"Frees":22},{"Size":480,"Mallocs":34,"Frees":21},{"Size":512,"Mallocs":951,"Frees":91},{"Size":576,"Mallocs":364,"Frees":122},{"Size":640,"Mallocs":532,"Frees":270},{"Size":704,"Mallocs":93,"Frees":39},{"Size":768,"Mallocs":83,"Frees":35},{"Size":896,"Mallocs":308,"Frees":175},{"Size":1024,"Mallocs":226,"Frees":122},{"Size":1152,"Mallocs":198,"Frees":100},{"Size":1280,"Mallocs":314,"Frees":171},{"Size":1408,"Mallocs":77,"Frees":47},{"Size":1536,"Mallocs":80,"Frees":54},{"Size":1792,"Mallocs":199,"Frees":107},{"Size":2048,"Mallocs":112,"Frees":48},{"Size":2304,"Mallocs":71,"Frees":32},{"Size":2688,"Mallocs":206,"Frees":81},{"Size":3072,"Mallocs":39,"Frees":15},{"Size":3200,"Mallocs":16,"Frees":7},{"Size":3456,"Mallocs":44,"Frees":29},{"Size":4096,"Mallocs":192,"Frees":83},{"Size":4864,"Mallocs":44,"Frees":25},{"Size":5376,"Mallocs":105,"Frees":43},{"Size":6144,"Mallocs":25,"Frees":5},{"Size":6528,"Mallocs":22,"Frees":7},{"Size":6784,"Mallocs":3,"Frees":0},{"Size":6912,"Mallocs":4,"Frees":2},{"Size":8192,"Mallocs":59,"Frees":10},{"Size":9472,"Mallocs":31,"Frees":12},{"Size":9728,"Mallocs":5,"Frees":2},{"Size":10240,"Mallocs":5,"Frees":0},{"Size":10880,"Mallocs":27,"Frees":11},{"Size":12288,"Mallocs":4,"Frees":1},{"Size":13568,"Mallocs":4,"Frees":2},{"Size":14336,"Mallocs":9,"Frees":2},{"Size":16384,"Mallocs":10,"Frees":2},{"Size":18432,"Mallocs":4,"Frees":2}]},
"warning_failed": 0,
"warning_generated": 0
}
```

If we find the DERP metrics useful we could consider how to include them in Prometheus scrapes based on the tailnet `varz` package.  That's for a later PR if at all.
2024-02-14 15:11:45 +04:00
Muhammad Atif Ali 53c55439be chore (examples/templates/incus): fix a typo (#12123) 2024-02-13 19:16:33 +00:00
Steven Masley 5d483a7ea1 fix: do not query user_link for deleted accounts (#12112) 2024-02-13 13:02:21 -06:00
Steven Masley 06f3ab1206 chore: add database test fixture to insert non-unique linked_ids (#12111)
* chore: add database test fixture to insert non-unique linked_ids
2024-02-13 12:06:47 -06:00
Kayla Washburn-Love d37b131426 feat: add activity status and autostop reason to workspace overview (#11987) 2024-02-13 10:50:17 -07:00
Muhammad Atif Ali e53d8bdb50 docs: update modules docs (#11911) 2024-02-13 15:35:09 +00:00
Cian Johnston 68641f9e2f chore(examples/templates/incus): fix incus group name in README (#12120) 2024-02-13 15:31:07 +00:00
dependabot[bot] e938690b1e chore: bump golang.org/x/mod from 0.14.0 to 0.15.0 (#12094)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-13 18:25:26 +03:00
Muhammad Danish 3c536aa880 ci: use repo secret for syncing winget-pkgs fork (#12108) 2024-02-13 18:25:13 +03:00
dependabot[bot] 28bbdee655 chore: bump github.com/go-playground/validator/v10 (#12096)
Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.17.0 to 10.18.0.
- [Release notes](https://github.com/go-playground/validator/releases)
- [Commits](https://github.com/go-playground/validator/compare/v10.17.0...v10.18.0)

---
updated-dependencies:
- dependency-name: github.com/go-playground/validator/v10
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-13 17:54:13 +03:00
dependabot[bot] 4760e85c15 chore: bump golang.org/x/net from 0.20.0 to 0.21.0 (#12097)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.21.0.
- [Commits](https://github.com/golang/net/compare/v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-13 17:53:16 +03:00
dependabot[bot] 9560d9a68b ci: bump the github-actions group with 2 updates (#12091)
Bumps the github-actions group with 2 updates: [crate-ci/typos](https://github.com/crate-ci/typos) and [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action).


Updates `crate-ci/typos` from 1.18.0 to 1.18.2
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.18.0...v1.18.2)

Updates `aquasecurity/trivy-action` from 0.16.1 to 0.17.0
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](https://github.com/aquasecurity/trivy-action/compare/d43c1f16c00cfd3978dde6c07f4bbcf9eb6993ca...84384bd6e777ef152729993b8145ea352e9dd3ef)

---
updated-dependencies:
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: aquasecurity/trivy-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-13 17:52:12 +03:00
Garrett Delfosse 3ab3a62bef feat: add port-sharing backend (#11939) 2024-02-13 09:31:20 -05:00
Cian Johnston c939416702 chore(examples): add sample Incus template (#12114)
Adds sample incus template created for FOSDEM 2024; there's enough intricacy involved to make it worth persisting
2024-02-13 14:30:31 +00:00
Dean Sheather e1e352d8c1 feat: add template activity_bump property (#11734)
Allows template admins to configure the activity bump duration. Defaults to 1h.
2024-02-13 07:00:35 +00:00
Dean Sheather fead57f304 fix: allow access to unhealthy/initializing apps (#12086) 2024-02-13 16:30:49 +10:00
Cian Johnston ec25fb8bbc fix(docs/networking/stun): convert svg diagrams to png 2024-02-12 17:27:53 +00:00
Cian Johnston 2fabc9499a fix(docs): remove inline mermaid diagrams (#12107) 2024-02-12 15:56:37 +00:00
Cian Johnston 1cc51b009a chore(examples): remove deprecated startup_script_timeout and shutdown_script_timeout (#12104)
Removes deprecated startup_script_timeout and shutdown_script_timeout from our example templates.

Co-authored-by: Muhammad Atif Ali <atif@coder.com>
2024-02-12 14:29:41 +00:00
Marcin Tojek 3e68650791 feat: support order property of coder_app resource (#12077) 2024-02-12 15:11:31 +01:00
Cian Johnston 1e9a3c952f chore(docs/networking/stun): fix diagram in section 2 (#12103) 2024-02-12 12:33:41 +00:00
Cian Johnston d1a522a8fc chore(docs): add requirements re ports and stun server to docs (#12026)
Adds documentation on port requirements and a short overview of STUN with some example scenarios.

Co-authored-by: Dean Sheather <dean@deansheather.com>
Co-authored-by: Spike Curtis <spike@coder.com>
2024-02-12 11:42:27 +00:00
Dean Sheather 2fc3064653 chore: add tests for app ID copy in app healths (#12088) 2024-02-12 05:49:48 +00:00
Colin Adler 06254a167f chore(docs): add v2.8.2 changelog (#12089) 2024-02-12 05:48:34 +00:00
Dean Sheather 429144da22 fix: copy app ID in healthcheck (#12087) 2024-02-12 05:01:16 +00:00
Eric Paulsen bb308851f5 docs: fix jetbrains reconnect faq (#12073)
* docs: fix jetbrains reconnect faq

* make: fmt

* add asher feedback
2024-02-09 23:44:33 +00:00
Bruno Quaresma 390217b396 feat(site): add create template from scratch (#12082) 2024-02-09 14:42:26 +00:00
Cian Johnston 2b307c7c4e fix(cli/server): do not redirect /healthz (#12080) 2024-02-09 13:44:47 +00:00
Spike Curtis 92b2e26a48 feat: send log limit exceeded in response, not error (#12078)
When we exceed the db-imposed limit of logs, we need to communicate that back to the agent.  In v1 we did it with a 4xx-level HTTP status, but with dRPC, the errors are delivered as strings, which feels fragile to me for something we want to gracefully handle.

So, this PR adds the log limit exceeded as a field on the response message, and fixes the API handler to set it as appropriate instead of an error.
2024-02-09 16:17:20 +04:00
Spike Curtis 1f5a6d59ba chore: consolidate websocketNetConn implementations (#12065)
Consolidates websocketNetConn from multiple packages in favor of a central one in codersdk
2024-02-09 11:39:08 +04:00
Colin Adler ec8e41f516 chore: add logging around agent app health reporting (#12071) 2024-02-08 23:37:44 -06:00
Marcin Tojek c0e169ebf9 feat: support custom order of agent metadata (#12066) 2024-02-08 17:29:34 +01:00
Mathias Fredriksson e659957b65 fix(cli/ssh): prevent reads/writes to stdin/stdout in stdio mode (#12045)
Fixes #11530
2024-02-08 13:09:42 +02:00
Spike Curtis 151aaadc23 fix: allow startup scripts larger than 32k (#12060)
Fixes #12057 and adds a regression test.
2024-02-07 22:26:42 +04:00
Bruno Quaresma 4d63a473b2 fix(site): fix infinity loading when template has no previous version (#12059) 2024-02-07 14:56:09 -03:00
Mathias Fredriksson 040ce40ed8 fix(dogfood): add ability to synchronize with startup script via done file (#12058) 2024-02-07 19:16:18 +02:00
Bruno Quaresma d8a8070986 fix(site): enable submit when auto start and stop are both disabled (#12055) 2024-02-07 14:06:48 -03:00
Bruno Quaresma 4b1bac31b6 feat(site): allow any file extension on template editor (#12000) 2024-02-07 13:24:28 -03:00
Marcin Tojek 4e7b208068 fix(site): e2e: print API backend calls (#12051) 2024-02-07 15:50:07 +01:00
Eric Paulsen 1abe0cfa1a docs: fix /audit & /insights params (#12043) 2024-02-07 08:38:54 -05:00
Spike Curtis 1cf4b62867 feat: change agent to use v2 API for reporting stats (#12024)
Modifies the agent to use the v2 API to report its statistics, using the `statsReporter` subcomponent.
2024-02-07 15:26:41 +04:00
Muhammad Atif Ali 70ad833b02 ci: fix GH_TOKEN in release.yaml (#12044) 2024-02-07 13:37:11 +03:00
Mathias Fredriksson f2aef0726b fix(agent/agentssh): allow scp to exit with zero status (#12028)
Fixes #11786
2024-02-07 10:22:31 +02:00
Josh Vawdrey d3ccb07361 feat(cli): support header and header-command in config-ssh (#10413)
Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2024-02-07 10:21:26 +02:00
Ben Potter d6cdaae8b1 docs: add v2.8.0 changelog (#12042)
* docs: add v2.8.0 changelog

* fmt
2024-02-07 00:14:17 +00:00
Cian Johnston 36808f19dc feat!: update terraform to version 1.6.x, relax max version constraint (#12027)
* feat(provisioner): relax max terraform version constraint

* feat!(scripts/Dockerfile.base): update bundled terraform to 1.6.x

* bump terraform version in Dogfood image

* fix over-zealous rename
2024-02-06 17:58:26 -06:00
Kayla Washburn-Love b8e32a37de fix: use replace when redirecting from /health (#12039)
`pushHistory` will break the back button, so we need to use `replaceHistory` instead
2024-02-06 14:27:32 -07:00
Marcin Tojek 3f04e98cfa feat(cli): pull templates in zip format (#12032) 2024-02-06 19:17:29 +01:00
Spike Curtis 213ae69bee fix: start timer before subscribing to avoid test race (#12031)
Fixes #12030

This is a good example of the kind of thing I'd like to address with a time-testing lib.  The problem is that there is a race between the watchdog starting it's timer and the test incrementing the time.  What would make this easier is if the time-testing library could wait for and assert the call to start the timer before incrementing the time.
2024-02-06 20:21:23 +04:00
Marcin Tojek b6806bca70 fix: nix: google-chrome installed conditionally (#12029) 2024-02-06 16:46:58 +01:00
Dean Sheather 98b86f3cd6 chore: add logs to pq notification dialer (#12020) 2024-02-06 15:21:48 +00:00
Spike Curtis e09cd2c6bd feat: add watchdog to pubsub (#12011)
adds a watchdog to our pubsub and runs it for Coder server.

If the watchdog times out, it triggers a graceful exit in `coder server` to give any provisioner jobs a chance to shut down.

c.f. #11950
2024-02-06 16:58:45 +04:00
Cian Johnston f1e5b4fbb8 ci: stop deploying legacy wsproxies (#12025) 2024-02-06 11:00:10 +00:00
Cian Johnston 26379877b2 fix(dogfood): stop overriding /etc/apt/sources.list with tsw mirrors (#11999) 2024-02-06 09:39:05 +00:00
Colin Adler c7f52b73bb feat(coderd): add prometheus metrics to servertailnet (#11988) 2024-02-05 23:57:18 -06:00
Spike Curtis c84a637116 fix: stop logging error on query canceled (#12017)
Fixes flake seen here: https://github.com/coder/coder/actions/runs/7782340530/job/21218566449
2024-02-06 08:43:34 +04:00
Kayla Washburn-Love b73e66e9a9 feat: show workspace name suggestions below the name field (#12001) 2024-02-05 10:40:15 -07:00
dependabot[bot] 52ec3edd5d ci: bump the github-actions group with 4 updates (#12019)
Bumps the github-actions group with 4 updates: [buildjet/cache](https://github.com/buildjet/cache), [crate-ci/typos](https://github.com/crate-ci/typos), [codecov/codecov-action](https://github.com/codecov/codecov-action) and [hmarr/auto-approve-action](https://github.com/hmarr/auto-approve-action).


Updates `buildjet/cache` from 3 to 4
- [Changelog](https://github.com/BuildJet/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/buildjet/cache/compare/v3...v4)

Updates `crate-ci/typos` from 1.17.2 to 1.18.0
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.17.2...v1.18.0)

Updates `codecov/codecov-action` from 3 to 4
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v3...v4)

Updates `hmarr/auto-approve-action` from 3 to 4
- [Release notes](https://github.com/hmarr/auto-approve-action/releases)
- [Commits](https://github.com/hmarr/auto-approve-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: buildjet/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: hmarr/auto-approve-action
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-05 14:46:05 +00:00
dependabot[bot] 1f0ba745e9 chore: bump github.com/bramvdbogaerde/go-scp (#12015)
Bumps [github.com/bramvdbogaerde/go-scp](https://github.com/bramvdbogaerde/go-scp) from 1.2.1-0.20221219230748-977ee74ac37b to 1.3.0.
- [Release notes](https://github.com/bramvdbogaerde/go-scp/releases)
- [Commits](https://github.com/bramvdbogaerde/go-scp/commits/v1.3.0)

---
updated-dependencies:
- dependency-name: github.com/bramvdbogaerde/go-scp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-05 14:50:02 +02:00
dependabot[bot] c1e01dfb7b chore: bump github.com/elastic/go-sysinfo from 1.11.0 to 1.12.0 (#12013)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-05 15:37:57 +03:00
Spike Curtis e5ba586e30 fix: fix graceful disconnect in DialWorkspaceAgent (#11993)
I noticed in testing that the CLI wasn't correctly sending the disconnect message when it shuts down, and thus agents are seeing this as a "lost" peer, rather than a "disconnected" one. 

What was happening is that we just used a single context for everything from the netconn to the RPCs, and when the context was canceled we failed to send the disconnect message due to canceled context.

So, this PR splits things into two contexts, with a graceful one set to last up to 1 second longer than the main one.
2024-02-05 14:01:37 +04:00
Spike Curtis bb99cb7d2b chore: move FakeCoordinator to tailnettest (#11992)
Moves FakeCoordinator to tailnettest since it's reused in testing multiple packages in this stack of PRs.
2024-02-05 13:49:32 +04:00
Spike Curtis 646ac942b2 chore: rename FakeCoordinator for export (#11991)
Part of a stack that fixes graceful disconnect from the CLI to tailnet.  I reuse FakeCoordinator in a test for graceful disconnects.
2024-02-05 13:33:31 +04:00
Eric Paulsen f57ce97b5a docs: add faq for gateway reconnects (#12007)
* docs: add faq for gateway reconnects

* make: fmt
2024-02-04 15:50:53 -06:00
Kayla Washburn-Love 1d14d4e58c fix: use dark background in terminal, even when a light theme is selected (#12004) 2024-02-02 15:05:52 -07:00
Jon Ayers 73c5993bea fix: only display xray results if vulns > 0 (#11989) 2024-02-02 11:02:46 -06:00
Cian Johnston 6593de3c73 fix(dogfood/flake.nix): add google-chrome (#11974) 2024-02-02 15:56:06 +00:00
Bruno Quaresma 9b930f8fad feat(site): show deprecation message on template page (#11996) 2024-02-02 14:13:35 +00:00
Bruno Quaresma 2e378b4894 fix(site): fix parameter input icon shrink (#11995) 2024-02-02 13:49:49 +00:00
Mathias Fredriksson aae228ac01 fix(dogfood): resolve module.git-clone.repo_dir containing ~/ (#11994) 2024-02-02 14:21:34 +02:00
Mathias Fredriksson bddea7bcf9 feat(cli/vscodessh): add support for --wait and scripts that block login (#10473) 2024-02-02 13:18:26 +02:00
Kayla Washburn-Love c6c71de353 fix: change build status colors (#11985) 2024-02-01 18:02:40 -07:00
dependabot[bot] efac9ced3e chore: bump github.com/moby/moby (#11975)
Bumps [github.com/moby/moby](https://github.com/moby/moby) from 24.0.1+incompatible to 25.0.2+incompatible.
- [Release notes](https://github.com/moby/moby/releases)
- [Commits](https://github.com/moby/moby/compare/v24.0.1...v25.0.2)

---
updated-dependencies:
- dependency-name: github.com/moby/moby
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-01 13:58:13 -06:00
Muhammad Atif Ali 21237d96a5 docs: update remote docker host docs (#11919)
* docs: update remote docker host docs

Adds a link to external provisioners as a method to use remote docker hosts

* `make fmt`

* Update docker.md

* fmt
2024-02-01 22:43:38 +03:00
Muhammad Atif Ali 9616b92f0e chore(dogfood): fix nix icon path (#11984) 2024-02-01 19:27:41 +00:00
Bruno Quaresma 96346525e0 fix(site): fix text overflow on batch ws deletion (#11981)
Before:
![image](https://github.com/coder/coder/assets/3165839/723a8fd7-8f63-4712-8af1-cd442455c723)

After:
<img width="674" alt="Screenshot 2024-02-01 at 13 48 56" src="https://github.com/coder/coder/assets/3165839/91c3099e-6a11-4beb-b46b-70a9a6c4abb4">
2024-02-01 14:02:08 -03:00
Marcin Tojek ad8e0db172 feat: add custom error message on signups disabled page (#11959) 2024-02-01 18:01:25 +01:00
Kayla Washburn-Love e070a55142 refactor: stabilize theme.roles (#11969) 2024-02-01 09:53:26 -07:00
Bruno Quaresma 6c9f60a9c5 refactor(site): only display quota if it is higher than 0 (#11979) 2024-02-01 13:49:48 -03:00
Steven Masley 79d5c238cc fix: always return a clean http client for promoauth (#11963)
* fix: add unit test to verify default client is not broken

* always return a clean http client
* No need to clone the tripper
2024-02-01 11:13:34 -05:00
Bruno Quaresma 1a94686928 refactor(site): add table chosmetic changes (#11977)
- Set default 14px as the default font size for the table content
- Add `xsmall` size for checkboxes
- Remove checkbox wrapper padding on the table heading

Before:
<img width="1512" alt="Screenshot 2024-02-01 at 10 22 10" src="https://github.com/coder/coder/assets/3165839/92b844ae-f2bf-476a-89fe-90b16f19c306">

After: 
<img width="1512" alt="Screenshot 2024-02-01 at 10 26 00" src="https://github.com/coder/coder/assets/3165839/0f87d098-4b13-4373-96d2-2c18ee2587f6">
2024-02-01 10:41:15 -03:00
Spike Curtis 1aa117b9ec chore: rename client Listen to ConnectRPC (#11916)
ConnectRPC seems more appropriate for this function
2024-02-01 14:44:11 +04:00
dependabot[bot] 1031ccb3c9 chore: bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 (#11968)
Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.5 to 1.1.12.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.12/CHANGELOG.md)
- [Commits](https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.12)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-01 13:15:01 +03:00
Spike Curtis d5a98cc6d7 fix: avoid race in TestPGPubsub_Metrics by using Eventually (#11973)
Annoyingly, prometheus Registry collects metrics async, which is causing our test to be racy.  They also don't export enough from the Metric interface for us to replicate a synchronous collect, so we have to use Eventually to test.
2024-02-01 12:10:19 +04:00
Spike Curtis 5a359d50dd feat: add metrics to PGPubsub (#11971)
Adds prometheus metrics to PGPubsub for monitoring its health and performance in production.

Related to #11950 --- additional diagnostics to help figure out what's happening
2024-02-01 11:25:03 +04:00
Asher e748312193 fix(dogfood): fix startup script looping (#11972)
Seems to be on account of the quotes interpreting a ~ literally.  We do
replace it with /home/coder but only if it matches ~/, not ~ alone.
2024-01-31 21:33:02 -09:00
Colin Adler 3ace7982aa fix: rewrite url to agent ip in single tailnet (#11810)
This restores previous behavior of being able to cache connections
across agents in single tailnet.
2024-02-01 00:25:52 -06:00
Spike Curtis 073d1f7078 chore: remove pingWebSocket since yamux runs keepalives (#11914)
Since we run yamux over the websocket, we don't need to ping at the websocket layer because yamux has a 30 second keepalive mechanism enabled in the default config.
2024-02-01 09:48:58 +04:00
Colin Adler 4ed1f5581a chore(coderd): add logging to agent rpc yamux conn (#11965) 2024-01-31 23:17:20 -06:00
Spike Curtis cc0dc103b6 chore: remove agentsdk client RPC() function (#11913)
The RPC() function isn't called, since Listen() was modified to do this job.

Listen() has the right signature, since it returns a drpc.Conn, rather than the Agent API.  That's because tailnet v2 and agent v2 are separate APIs served over the same connection.

It might be clearer to rename `Listen()` to `RPC()` but I'll save that for a different PR.
2024-02-01 08:22:12 +04:00
Spike Curtis eb03e4490a feat: add statsReporter for reporting stats on agent v2 API (#11920)
Adds a new statsReporter subcomponent of the agent, which in a later PR will be used to report stats over the v2 API.

Refactors the logic a bit so that we can handle starting and stopping stats reporting if the agent API connection drops and reconnects.
2024-02-01 08:21:01 +04:00
Spike Curtis b79785c86f feat: move agent v2 API connection monitoring to yamux layer (#11910)
Moves monitoring of the agent v2 API connection to the yamux layer.

Present behavior monitors this at the websocket layer, and closes the websocket on completion. This can cause yamux to hit unexpected errors since the connection is closed underneath it.

This might be the cause of yamux errors that some customers are seeing

![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/tCz4CxRU9jhAJ7zH8RTi/53b8b5ef-e9e5-44a5-b559-99c37c136071.png)

In any case, it's more graceful to close yamux first and let yamux close the underlying websocket.  That should limit yamux error logging to truly unexpected/error cases.

The only downside is that the yamux `Close()` doesn't accept a reason, so if the agent becomes outdated and we close the API connection, the agent just sees the connection close without a reason.  I'm not sure we log this at the agent anyway, but it would be nice.  I think more accurate logging on Coderd are more important.

I've also added some logging when the monitor disconnects for reasons other than the context being canceled (e.g. agent outdated, failed pings).
2024-02-01 08:18:35 +04:00
Spike Curtis 13e214f7f1 feat: add logging to agent yamux session (#11912)
Log yamux errors and warnings in the agent.
2024-02-01 08:18:13 +04:00
Michael Smith b0a855caa4 fix: improve click UX and styling for Auth Token page (#11863)
* wip: commit progress for clipboard update

* wip: push more progress

* chore: finish initial version of useClipboard revamp

* refactor: update API query to use newer RQ patterns

* fix: update importers of useClipboard

* fix: increase clickable area of CodeExample

* fix: update styles for CliAuthPageView

* fix: resolve issue with ref re-routing

* docs: update comments for clarity

* wip: commit progress on clipboard tests

* chore: add extra test case for referential stability

* wip: disable test stub to avoid breaking CI

* wip: add test case for tab-switching

* feat: finish changes

* fix: improve styling for strong text

* fix: make sure period doesn't break onto separate line

* fix: make center styling more friendly to screen readers

* refactor: clean up mocking implementation

* fix: resolve security concern for clipboard text

* fix: update CodeExample to obscure text when appropriate

* fix: apply secret changes to relevant code examples

* refactor: simplify code for obfuscating text

* fix: partially revert clipboard changes

* fix: clean up page styling further

* fix: remove duplicate property identifier

* refactor: rename variables for clarity

* fix: simplify/revert CopyButton component design

* fix: update how dummy input is hidden from page

* fix: remove unused onClick handler prop

* fix: resolve unused import

* fix: opt code examples out of secret behavior
2024-01-31 21:25:30 -05:00
Colin Adler c7f51a9d70 chore(site): update time until shutdown tooltip language (#11964) 2024-01-31 16:21:35 -06:00
Kayla Washburn-Love d2e6405322 chore: add inactive role to experimental theme (#11967) 2024-01-31 15:16:17 -07:00
Bruno Quaresma 4df913372f feat(site): display xray scan result in the agent (#11955) 2024-01-31 19:16:01 -03:00
Steven Masley ac64155282 fix: strip timezone information from a date in dau response (#11962)
* fix: strip timezone information from a date in dau response

Timezone information is lost, so do not forward it to the client.

* fix: timezone offset should be flipped
* Make tests deterministic
2024-01-31 16:01:50 -06:00
Kayla Washburn-Love 76e73287a5 refactor: add modules/templates and modules/workspaces (#11947) 2024-01-31 12:09:36 -07:00
Muhammad Atif Ali 4604db072a fix(dogfood): fix startup script on workspace creation (#11958) 2024-01-31 18:01:22 +03:00
Muhammad Atif Ali d2b4d58e96 chore(dogfood): use better names for image options (#11957) 2024-01-31 14:36:01 +00:00
Muhammad Atif Ali 215a9d1b30 chore: experiment building dogfood image with nix (#11680) 2024-01-31 14:27:11 +00:00
Marcin Tojek 13cbca679e feat: support template bundles as zip archives (#11839) 2024-01-31 14:49:55 +01:00
Mathias Fredriksson b25deaae20 fix(coderd/database): fix limit in GetUserWorkspaceBuildParameters (#11954) 2024-01-31 13:56:36 +02:00
Spike Curtis a34cada09a feat: add logging to pgPubsub (#11953)
Should be helpful for #11950

Adds a logger to pgPubsub and logs various events, most especially connection and disconnection from postgres.
2024-01-31 15:49:16 +04:00
Spike Curtis 1c8b803785 feat: add logging to pgcoord subscribe/unsubscribe (#11952)
Adds logging to unsubscribing from peer and tunnel updates in pgcoordinator, since #11950 seems to be problem with these subscriptions
2024-01-31 12:15:58 +04:00
Jon Ayers 0c30dde9b5 feat: add customizable upgrade message on client/server version mismatch (#11587) 2024-01-30 17:11:37 -06:00
Ammar Bandukwala adbb025e74 feat: add user-level parameter autofill (#11731)
This PR solves #10478 by auto-filling previously used template values in create and update workspace flows.

I decided against explicit user values in settings for these reasons:

* Autofill is far easier to implement
* Users benefit from autofill _by default_ — we don't need to teach them new concepts
* If we decide that autofill creates more harm than good, we can remove it without breaking compatibility
2024-01-30 16:02:21 -06:00
Kayla Washburn-Love aeb4112513 chore: update storybook (#11936) 2024-01-30 14:23:40 -07:00
Spike Curtis 520b12e1a2 fix: close MultiAgentConn when coordinator closes (#11941)
Fixes an issue where a MultiAgentConn isn't closed properly when the coordinator it is connected to is closed.

Since servertailnet checks whether the conn is closed before reinitializing, it is important that we check this, otherwise servertailnet can get stuck if the coordinator closes (e.g. when we switch from AGPL to PGCoordinator after decoding a license).
2024-01-31 00:38:19 +04:00
Colin Adler 2fd1a726aa fix: only delete expired agents on success (#11940) 2024-01-30 14:11:45 -06:00
Colin Adler 27f3b7a814 fix: add timeout to listening ports request (#11935)
This can potentially hang for 15m if the agent is unreachable.
2024-01-30 13:53:52 -06:00
Bruno Quaresma 7f1c808ff9 feat(site): simplify create template form by removing advanced settings (#11918) 2024-01-30 16:40:59 -03:00
Kayla Washburn-Love 619bdd1e7a refactor: redesign Paywall component (#11907) 2024-01-30 10:26:19 -07:00
Kayla Washburn-Love 20dcefa156 add an interaction test to InfoTooltip (#11905) 2024-01-30 10:20:11 -07:00
Bruno Quaresma e26ba1affd feat(site): do not show popover on update deadline (#11921) 2024-01-30 14:11:15 -03:00
Bruno Quaresma dcab6fa5a4 feat(site): display user avatar (#11893)
* add owner API to workspace and workspace build responses
* display user avatar in workspace top bar

Co-authored-by: Cian Johnston <cian@coder.com>
2024-01-30 17:07:06 +00:00
Mathias Fredriksson 83eea2d323 feat(scaletest/templates): add support for concurrent scenarios (#11753) 2024-01-30 14:54:54 +02:00
Bruno Quaresma 4b27c77969 fix(site): fix parameters' request upon template variables update (#11898)
Fix https://github.com/coder/coder/issues/11870
2024-01-30 08:03:53 -03:00
Mathias Fredriksson 60653bbacb fix(cli): allow template name length of 32 in template push and create (#11915) 2024-01-30 12:47:10 +02:00
Muhammad Atif Ali 86e33257af chore(docs): fix a typo (#11895) 2024-01-30 12:00:25 +03:00
Spike Curtis 0fc177203e feat: use agent v2 API to update app health (#11889)
Use the Agent v2 API to update App Health
2024-01-30 11:35:12 +04:00
Spike Curtis 2599850e54 feat: use agent v2 API to post startup (#11877)
Uses the v2 Agent API to post startup information.
2024-01-30 11:23:28 +04:00
Spike Curtis da8bb1c198 feat: use agent v2 API to fetch manifest (#11832)
Agent uses the v2 API to obtain the manifest, instead of the HTTP API.
2024-01-30 10:11:28 +04:00
Spike Curtis 9cf4e7f15a fix: prevent agent_test.go from failing on error logs (#11909)
We're failing tests on error logs like this: https://github.com/coder/coder/actions/runs/7706053882/job/21000984583

Unfortunately, the error we hit, when the underlying connection is closed, is unexported, so we can't specifically ignore it.

Part of the issue is that agent.Close() doesn't wait for these goroutines to complete before returning, so the test harness proceeds to close the connection. This looks to our product code like the network connection failing.  It would be possible to fix this, but just doesn't seem worth it for the extra insurance of catching other error logs in these tests.
2024-01-30 10:04:01 +04:00
Spike Curtis d3983e4dba feat: add logging to client tailnet yamux (#11908)
Adds logging to yamux when used for tailnet client connections, e.g. CLI and wsproxy.  This could be useful for debugging connection issues with tailnet v2 API.
2024-01-30 09:58:59 +04:00
Spike Curtis 0eff646c31 chore: move proto to sdk conversion to agentsdk (#11831)
`agentsdk` depends on `agent/proto` because it needs to get the version to dial.

Therefore, the conversion routines need to live in `agentsdk` so that we can convert to and from the Manifest.

I briefly considered refactoring the agent to only reference `proto.Manifest`, but decided against it because we might have multiple protocol versions in the future, its useful to have a protocol-independent data structure.
2024-01-30 09:04:56 +04:00
Spike Curtis 1e8a9c09fe chore: remove legacy wsconncache (#11816)
Fixes #8218

Removes `wsconncache` and related "is legacy?" functions and API calls that were used by it.

The only leftover is that Agents still use the legacy IP, so that back level clients or workspace proxies can dial them correctly.

We should eventually remove this: #11819
2024-01-30 07:56:36 +04:00
Spike Curtis 13e24f21e4 feat: use Agent v2 API for Service Banner (#11806)
Agent uses the v2 API for the service banner, rather than the v1 HTTP API.

One of several for #10534
2024-01-30 07:44:47 +04:00
Jon Ayers 4f5a2f0a9b feat: add backend for jfrog xray support (#11829) 2024-01-29 19:30:02 -06:00
dependabot[bot] 46d92dac57 ci: bump the github-actions group with 5 updates (#11890)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:38:59 +03:00
dependabot[bot] 5937027c86 chore: bump github.com/gohugoio/hugo from 0.121.2 to 0.122.0 (#11883)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:36:23 +03:00
dependabot[bot] 4dc6a302f2 chore: bump google.golang.org/grpc from 1.60.1 to 1.61.0 (#11885)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:36:12 +03:00
dependabot[bot] 3b65a1508c chore: bump github.com/google/uuid from 1.5.0 to 1.6.0 (#11886)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:36:01 +03:00
dependabot[bot] 71b79eace4 chore: bump alpine from 3.19.0 to 3.19.1 in /scripts (#11887)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-30 00:35:44 +03:00
Eric Paulsen d8a3ebef31 docs: fix example template README 404s and semantics (#11903)
* docs: fix example template README 404s and semantics

* make: gen
2024-01-29 21:34:12 +00:00
Kayla Washburn-Love f572e18144 fix: fix cliui prompt styling (#11899) 2024-01-29 13:56:43 -07:00
Spike Curtis 207328ca50 feat: use appearance.Fetcher in agentapi (#11770)
This PR updates the Agent API to use the appearance.Fetcher, which is set by entitlement code in Enterprise coderd.

This brings the agentapi into compliance with the Enterprise feature.
2024-01-29 21:22:50 +04:00
Garrett Delfosse f54278cdfe fix: respect wait flag on ping (#11896) 2024-01-29 11:50:35 -05:00
Colin Adler bc14e926d8 feat: add option to speedtest to dump a pcap of network traffic (#11848) 2024-01-29 09:57:31 -06:00
Spike Curtis b2bc3fff33 fix: wait for new template version before promoting (#11874)
Fixes a test flake due to not waiting for the correct template version prior to promoting it.
2024-01-29 19:29:56 +04:00
Steven Masley 04a23261e6 chore: ensure github uids are unique (#11826) 2024-01-29 09:13:46 -06:00
Steven Masley d66e6e78ee fix: always attempt external auth refresh when fetching (#11762) (#11830)
* fix: always attempt external auth refresh when fetching
* refactor validate to check expiry when considering "valid"
2024-01-29 08:55:15 -06:00
Cian Johnston eeef56a655 feat(cli): show workspace favorite status in list output (#11878) 2024-01-29 14:14:12 +00:00
Cian Johnston 9abf6ec170 feat(site): show favorite workspaces in ui (#11875)
* Add Star beside workspace name to indicate favorite status in WorkspacesList
* Add button in workspace top row to toggle workspace favorite status
2024-01-29 13:39:31 +00:00
Bruno Quaresma acd22b2c65 fix(site): fix capitalized username (#11891)
Fix #11888
2024-01-29 10:24:19 -03:00
Mathias Fredriksson 3e89ba23e5 test(scaletest): fix websocket error during close (#11879)
Fixes #11735
2024-01-29 13:42:30 +02:00
Muhammad Atif Ali 8398b4188b ci: fix winget-release workflow (#11865) 2024-01-29 13:49:20 +03:00
Spike Curtis bc4ae53261 chore: refactor Appearance to an interface callable by AGPL code (#11769)
The new Agent API needs an interface for ServiceBanners, so this PR creates it and refactors the AGPL and Enterprise code to achieve it.

Before we depended on the fact that the HTTP endpoint was missing to serve an empty ServiceBanner on AGPL deployments, but that won't work with dRPC, so we need a real interface to call.
2024-01-29 12:17:31 +04:00
Marcin Tojek aacb4a2b4c feat: use map instead of slice in metrics aggregator (#11815) 2024-01-29 09:12:41 +01:00
Spike Curtis 37e9479815 fix: fix TestServiceBanners/Agent (#11768)
The original test is bugged in that it

1. creates a new AGPL coderd with a new database, so no appearance is set in the DB.
2. overwrites the agentClient so the assertion after removing the license is against the AGPL coderd
2024-01-29 11:56:33 +04:00
Spike Curtis f9fdd44510 feat: change codersdk to use tailnet v2 for DERPMap updates (#11736)
fixes #10533


refactors `codersdk` workspace agent dialer to use a single websocket connection to the tailnet v2 API for both coordination and DERPMap updates, rather than separate websockets (and the v1 API for DERPMaps).
2024-01-29 11:26:50 +04:00
Muhammad Atif Ali 699a4b8dd4 chore(dogfood): use built-in VS Code Desktop button over the module (#11869) 2024-01-29 00:37:22 +03:00
Eric Paulsen be4d5221ba docs: add guide for azure federation (#11864)
* docs: add guide for azure federation

* make: fmt

* refactor: arm secrets and semantics
2024-01-28 15:51:11 -05:00
Muhammad Atif Ali 2f9bf1ebe1 ci: validate template before pushing (#11867) 2024-01-27 10:02:10 +03:00
Spike Curtis 4825b7ccd2 fix: use new context after t.Parallel in TestOAuthAppSecrets
c.f. https://coder.com/blog/go-testing-contexts-and-t-parallel

fixes flakes like https://github.com/coder/coder/runs/20856469613
2024-01-27 08:45:43 +04:00
Muhammad Atif Ali de6d4794dc chore(dogfood): replace repo_dir with base_repo_dir in git-clone module (#11835)
Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2024-01-27 01:45:07 +00:00
Kayla Washburn-Love bb8ce7bc02 chore: move components/Resources to modules/resources (#11852) 2024-01-26 15:11:07 -07:00
Kayla Washburn-Love 8f46beef72 chore: remove most usage of PropsWithChildren (#11859) 2024-01-26 12:04:22 -07:00
Kayla Washburn-Love c2b6e204f3 fix: type error from theme update (#11844) 2024-01-26 10:56:19 -07:00
Kayla Washburn-Love c85fc3c8fe refactor: add more fill variants to the experimental theme (#11827) 2024-01-26 10:28:59 -07:00
Cian Johnston 42e997d39e fix(coderd/rbac): do not cache context cancellation errors (#11840)
#7439 added global caching of RBAC results.
Calls are cached based on hash(subject, object, action).
We often use dbauthz.AsSystemRestricted to handle "internal" authz calls, and these are often repeated with similar arguments and are likely to get cached.
So a transient error doing an authz check on a system function will be cached for up to a minute.
I'm just starting off with excluding context.Canceled but there's likely a whole suite of different errors we want to also exclude from the global cache.
2024-01-26 16:19:55 +00:00
Spike Curtis d6baa3cab0 fix: stop running tests that exec sh scripts in parallel (#11834)
Ok, so my last attempt at a fix here failed

https://github.com/coder/coder/actions/runs/7666229961/job/20893608286

I have a new theory: it's not the `terraform` binary that's busy, it's actually `fake_cancel.sh` and it gets marked busy when we `exec` it from the script we write.

Use of `exec` also replaces the executing code in place, rather than starting a new process/shell, so that's why the error we get says `terraform` is busy.
2024-01-26 19:22:35 +04:00
Bruno Quaresma 0ba035a16d refactor(site): improve parameters field (#11802) 2024-01-26 11:31:52 -03:00
Kira Pilot 4c71cccbc3 fix(site): disable autostart and autostop according to template settings (#11809)
* fix (site): disable autostart and autostop according to template settings

* checking form values again; wrote tests

* fixed closure and label bugs

* fix broken query key

* tweaks
2024-01-26 09:06:01 -05:00
Mathias Fredriksson 52c08a98bb test(scaletest): fix worksapcebuild retry (#11836) 2024-01-26 15:49:19 +02:00
Mathias Fredriksson 02124758fb feat(cli/exp): extend scaletest create-workspaces with --retry option (#11825)
Part of #11801
2024-01-26 11:29:48 +00:00
Cian Johnston fdf9f03097 fix(enterprise/cli): add ID to default columns in licenses list output (#11823) 2024-01-26 09:55:16 +00:00
Dean Sheather 29707099d7 chore: add agentapi tests (#11269) 2024-01-26 07:04:19 +00:00
Muhammad Atif Ali 541154b74b docs: simplify JFrog integration docs (#11787) 2024-01-25 19:50:06 -05:00
Steven Masley 005c014f13 chore: instrument additional github api calls (#11824)
* chore: instrument additional githubapi calls

This only affects github as a login source, not external auth.
2024-01-25 18:34:46 -06:00
Bruno Quaresma e371716b38 refactor(site): add minor workspace improvements (#11822) 2024-01-25 21:05:29 -03:00
Kayla Washburn-Love 73a6899f2c chore: miscellaneous cleanup (#11785) 2024-01-25 14:22:52 -07:00
Ammar Bandukwala 79568bf628 Revert "fix: always attempt external auth refresh when fetching (#11762)"
This reverts commit 0befc0826a.
2024-01-25 14:22:47 -06:00
Steven Masley 0befc0826a fix: always attempt external auth refresh when fetching (#11762)
* fix: always attempt external auth refresh when fetching
* refactor validate to check expiry when considering "valid"
2024-01-25 10:54:56 -06:00
Bruno Quaresma fd7f85bc5e fix(site): fix proxy settings link (#11817) 2024-01-25 12:16:24 +00:00
Cian Johnston 8eae4f83bf fix(coderd/provisionerdserver): fix test flake in TestHeartbeat (#11808) 2024-01-25 12:05:57 +00:00
Muhammad Atif Ali 979a920832 docs: use coder modules in offline deployments (#11788)
* docs: use coder modules in offline deployments

* fix typos

* Update offline installation instructions with Artifactory support for Coder modules

* Review suggestions
2024-01-25 08:01:56 +03:00
Ben Potter 6b0e1291d2 docs: add v2.7.3 changelog (#11811)
* docs: add v2.7.1 changelog

* docs: add v2.7.2 changelog
2024-01-24 16:53:08 -06:00
Kayla Washburn-Love 3d76e1b55c chore: clean up package.json and tsconfig (#11757) 2024-01-24 13:53:44 -07:00
Cian Johnston ecae6f9135 fix(enterprise/tailnet): handle query canceled error in sendBeat() (#11794) 2024-01-24 18:42:05 +00:00
Bruno Quaresma 8bc91b489e refactor(site): increase form fields gap (#11803) 2024-01-24 14:16:42 -03:00
Marcin Tojek 560e8cc1ae fix: check update permission to start workspace (#11798) 2024-01-24 17:18:03 +01:00
Cian Johnston 4616ccf462 fix(coderd): alter return signature of convertWorkspace, add check for requesterID (#11796) 2024-01-24 14:13:14 +00:00
Cian Johnston 70dc282b7d feat(cli): add favorite/unfavorite commands (#11793) 2024-01-24 14:05:39 +00:00
Cian Johnston f92336c4d5 feat(coderd): allow workspace owners to mark workspaces as favorite (#11791)
- Adds column `favorite` to workspaces table
- Adds API endpoints to favorite/unfavorite workspaces
- Modifies sorting order to return owners' favorite workspaces first
2024-01-24 13:39:19 +00:00
Bruno Quaresma 6145da8a9e refactor(site): verify external auth before display ws form (#11777) 2024-01-24 09:45:22 -03:00
Spike Curtis 5cbb76b47a fix: stop spamming DERP map updates for equivalent maps (#11792)
Fixes 2 related issues:

1. wsconncache had incorrect logic to test whether to send DERPMap updates, sending if the maps were equivalent, instead of if they were _not equivalent_.
2. configmaps used a bugged check to test equality between DERPMaps, since it contains a map and the map entries are serialized in random order. Instead, we avoid comparing the protobufs and instead depend on the existing function that compares `tailcfg.DERPMap`. This also has the effect of reducing the number of times we convert to and from protobuf.
2024-01-24 16:27:15 +04:00
Spike Curtis f5dbc718a7 fix: accept agent RPC connection without version query parameter (#11790)
Fixes an issue where Coder v2.7.1 agents connect to /api/v2/workspaceagents/me/rpc without a version query parameter
2024-01-24 09:10:16 +04:00
Colin Adler 13beb04521 fix: disable keepalives in workspaceapps transport (#11789)
Connection caching causes requests to hit the wrong workspaces. See
comment.

Fixes https://github.com/coder/coder/issues/11767
2024-01-24 14:46:59 +10:00
Muhammad Atif Ali 1e2634d2d0 chore(dogfood): use versioning for coder modules (#11774) 2024-01-24 01:33:25 +00:00
Kayla Washburn-Love 31a6a5dc6d chore: add stories for DropdownArrow (#11764) 2024-01-23 16:02:57 -07:00
Jon Ayers 383eed93f8 fix: use correct logger for lifecycle_executor (#11763) 2024-01-23 14:33:55 -06:00
Bruno Quaresma e828daba6e refactor(site): simplify create workspace form (#11771)
This is the first PR of a series of PRs trying to simplify and improve the create workspace flow.
- Use the existent template header and remove the selected template card
- Move the owner field to the general section so we don't have "anemic" sections with single fields

Before:
<img width="1512" alt="Screenshot 2024-01-23 at 10 22 45" src="https://github.com/coder/coder/assets/3165839/6a2ba6b4-9ffb-4576-9282-7901691f45ee">

Now:
<img width="1512" alt="Screenshot 2024-01-23 at 10 22 56" src="https://github.com/coder/coder/assets/3165839/84301548-4af9-4de0-96ff-2a6363fc8cf7">
2024-01-23 15:39:23 -03:00
Steven Masley d6ba0dfecb feat: add "updated" search param to workspaces (#11714)
* feat: add "updated" search param to workspaces
* rego -> sql needs to specify which <table>.organization_id
2024-01-23 11:52:06 -06:00
Steven Masley 081fbef097 fix: code-server path based forwarding, defer to code-server (#11759)
Do not attempt to construct a path based port forward url.
Always defer to code server, as it has it's own proxy method.
2024-01-23 11:36:44 -06:00
Marcin Tojek 77a4792ecd fix(cli): ssh: auto-update workspace (#11773) 2024-01-23 18:01:44 +01:00
Bruno Quaresma 369821ea19 feat(site): generates unique workspace names by default (#11772) 2024-01-23 15:55:29 +00:00
Bruno Quaresma 910f17f4e7 refactor(site): refactor external auth component (#11758)
Recommended improvements:
- Rename component for clarity 
- Simplify interface for contextual relevance 
- Handle polling errors based on section, not every button

Before:
<img width="1511" alt="Screenshot 2024-01-22 at 15 24 26" src="https://github.com/coder/coder/assets/3165839/cfb8c0bc-f5a2-4708-bd97-fdfc46bd1eee">

Now:
<img width="1512" alt="Screenshot 2024-01-22 at 15 24 41" src="https://github.com/coder/coder/assets/3165839/5aaad448-1bb2-45ea-9250-cd374a072be2">
2024-01-23 12:26:12 -03:00
Spike Curtis 059e533544 feat: agent uses Tailnet v2 API for DERPMap updates (#11698)
Switches the Agent to use Tailnet v2 API to get DERPMap updates.

Subsequent PRs will do the same for the CLI (`codersdk`) and `wsproxy`.
2024-01-23 14:42:07 +04:00
Spike Curtis 3e0e7f8739 feat: check agent API version on connection (#11696)
fixes #10531

Adds a check for `version` on connection to the Agent API websocket endpoint.  This is primarily for future-proofing, so that up-level agents get a sensible error if they connect to a back-level Coderd.

It also refactors the location of the `CurrentVersion` variables, to be part of the `proto` packages, since the versions refer to the APIs defined therein.
2024-01-23 14:27:49 +04:00
Spike Curtis eb12fd7d92 feat: make ServerTailnet set peers lost when it reconnects to the coordinator (#11682)
Adds support to `ServerTailnet` to set all peers lost before attempting to reconnect to the coordinator. In practice, this only really affects `wsproxy` since coderd has a local connection to the coordinator that only goes down if we're shutting down or change licenses.
2024-01-23 13:17:56 +04:00
dependabot[bot] f86186eef2 chore: bump github.com/hashicorp/terraform-json from 0.20.0 to 0.21.0 (#11738)
Bumps [github.com/hashicorp/terraform-json](https://github.com/hashicorp/terraform-json) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/hashicorp/terraform-json/releases)
- [Commits](https://github.com/hashicorp/terraform-json/compare/v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/terraform-json
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-23 09:46:11 +03:00
Ben Potter 18d43405c0 chore: change SSH wording on workspace page (#11702) 2024-01-23 09:45:44 +03:00
dependabot[bot] ca38bfd2fc ci: bump the github-actions group with 2 updates (#11745)
Bumps the github-actions group with 2 updates: [crate-ci/typos](https://github.com/crate-ci/typos) and [toshimaru/auto-author-assign](https://github.com/toshimaru/auto-author-assign).


Updates `crate-ci/typos` from 1.17.1 to 1.17.2
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.17.1...v1.17.2)

Updates `toshimaru/auto-author-assign` from 2.0.1 to 2.1.0
- [Release notes](https://github.com/toshimaru/auto-author-assign/releases)
- [Changelog](https://github.com/toshimaru/auto-author-assign/blob/main/CHANGELOG.md)
- [Commits](https://github.com/toshimaru/auto-author-assign/compare/v2.0.1...v2.1.0)

---
updated-dependencies:
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: toshimaru/auto-author-assign
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-23 09:43:02 +03:00
Stephen Kirby 91a8b1b886 chore: fix broken docs links (#11760)
* fixed broken install/packages.md link

* fixed broken docs links

* fixed admin/auth link

* fixed example-guide links

* replaced mitchell tweet with nix docs

* make fmt

* replaced private image with imgur
2024-01-22 18:45:34 -06:00
Asher 3014777d2a feat: add endpoints to oauth2 provider applications (#11718)
These will show up when configuring the application along with the
client ID and everything else.  Should make it easier to configure the
application, otherwise you will have to go look up the URLs in the
docs (which are not yet written).

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
2024-01-22 13:25:25 -09:00
Steven Masley 8e0a153725 chore: implement device auth flow for fake idp (#11707)
* chore: implement device auth flow for fake idp
2024-01-22 20:46:05 +00:00
Asher 16c6cefde8 chore: pass lifetime directly into api key generate (#11715)
Rather than passing all the deployment values.  This is to make it
easier to generate API keys as part of the oauth flow.

I also added and fixed a test for when the lifetime is set and the
default and expiration are unset.

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
2024-01-22 11:42:55 -09:00
Bruno Quaresma a31d19d538 refactor(site): apply cosmetic changes and remove ExternalAuth from settings page (#11756) 2024-01-22 16:07:43 -03:00
Asher 7589df325b fix: display error when fetching OAuth2 provider apps (#11713) 2024-01-22 09:56:36 -09:00
Kayla Washburn-Love 69e963b1a2 refactor: move dashboard functionality to modules/dashboard/ (#11721) 2024-01-22 11:44:33 -07:00
Bruno Quaresma 14f114b224 chore(site): add test for sensitive value (#11755) 2024-01-22 15:03:15 -03:00
Kayla Washburn-Love f74ef142d0 refactor: reorganize auth components and hooks (#11717) 2024-01-22 10:43:32 -07:00
Bruno Quaresma f02561a599 chore(site): minor refactor to the resource metadata code (#11746) 2024-01-22 12:55:46 -03:00
Spike Curtis 5388a1b6d7 fix: use TSMP ping for reachability, not latency (#11749)
Use TSMP ping for reachability, but leave Disco ping for when we call Ping() since we often use that to determine whether we have a direct connection.

Also adds unit tests to make sure Ping() returns direct connection vs DERP correctly.
2024-01-22 17:37:15 +04:00
Ben Potter 66f119bde8 docs: add v2.7.1 changelog (#11747) 2024-01-22 07:09:18 -06:00
Spike Curtis 7ffd99cfe2 fix: use DiscoPing (partially reverts #11306) (#11744) 2024-01-22 12:40:21 +00:00
Spike Curtis 3d85cdfa11 feat: set peers lost when disconnected from coordinator (#11681)
Adds support to Coordination to call SetAllPeersLost() when it is closed. This ensure that when we disconnect from a Coordinator, we set all peers lost.

This covers CoderSDK (CLI client) and Agent.  Next PR will cover MultiAgent (notably, `wsproxy`).
2024-01-22 15:26:20 +04:00
Danny Kopping 9f6b38ce9c chore: use correct anchor link on scale.md (#11728) 2024-01-22 10:34:38 +00:00
Dean Sheather 15a90f028e chore: collect more template telemetry to gauge feature usage
We don't have visibility into some feature usage, so this adds a lot of fields missing from `database.Template` to `telemetry.Template`. Deprecation message is not collected, just whether it's set or not.
2024-01-22 18:55:27 +10:00
Spike Curtis b7b936547d feat: add setAllPeersLost to the configMaps subcomponent (#11665)
adds setAllPeersLost to the configMaps subcomponent of tailnet.Conn --- we'll call this when we disconnect from a coordinator so we'll eventually clean up peers if they disconnect while we are retrying the coordinator connection (or we don't succeed in reconnecting to the coordinator).
2024-01-22 12:12:15 +04:00
Spike Curtis f01cab9894 feat: use tailnet v2 API for coordination (#11638)
This one is huge, and I'm sorry.

The problem is that once I change `tailnet.Conn` to start doing v2 behavior, I kind of have to change it everywhere, including in CoderSDK (CLI), the agent, wsproxy, and ServerTailnet.

There is still a bit more cleanup to do, and I need to add code so that when we lose connection to the Coordinator, we mark all peers as LOST, but that will be in a separate PR since this is big enough!
2024-01-22 11:07:50 +04:00
Muhammad Atif Ali 5a2cf7cd14 chore(docs): remove tabs from appearance settings (#11726) 2024-01-20 13:27:28 +00:00
dependabot[bot] 83013792b1 chore: bump vite from 4.5.1 to 4.5.2 in /site (#11723)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-20 12:19:26 +00:00
Muhammad Atif Ali 4bed9611a8 fix(docs): fix tabs for support links (#11724) 2024-01-20 15:12:07 +03:00
Ben Potter 77de24c94f docs: add v2.7.0 changelog (#11719)
* docs: add v2.7.0 changelog

* some modifications
2024-01-19 23:11:35 +00:00
Kayla Washburn-Love 80eac73ed1 chore: remove useLocalStorage hook (#11712) 2024-01-19 16:04:19 -07:00
Asher fa99f6a200 chore: make yarn use the right version of node (#11716)
Otherwise if for example you try to run `yarn storybook` it complains
that the version of Node is wrong.

`pnpm storybook` works fine and that is probably what we should
actually use, but as long as we are installing Yarn and not restricting
its use we might as well make it use the right version of Node.
2024-01-19 12:59:38 -09:00
Kayla Washburn-Love 76911f1375 chore: fix TemplateVersionEditor story (#11709) 2024-01-19 12:13:30 -07:00
Steven Masley ca48b8783b fix: update template with noop returned undefined template (#11688)
* fix: doing a noop patch to templates resulted in 404

The patch response did not include the template. The UI required the
template to be returned to form the new page path

null is more explicit, and harder to make occur by mistake.
2024-01-19 18:54:25 +00:00
Kayla Washburn-Love 75d70a9542 chore: add a story for WorkspaceOutdatedTooltip (#11695) 2024-01-19 11:41:18 -07:00
Muhammad Atif Ali 6090007708 docs: update docs to set SupportLinks (#11699) 2024-01-19 20:10:10 +03:00
Steven Masley d67c9d1bb5 fix: set request header before do (#11706) 2024-01-19 16:14:08 +00:00
Steven Masley ccfd1a561b chore: improve device handling error message (#11606) 2024-01-19 09:41:52 -06:00
Mathias Fredriksson 593a1e9f60 feat(cli/exp): add target workspace/users to scaletest commands (#11701) 2024-01-19 15:32:46 +00:00
Marcin Tojek 4b059c4c93 fix: make workspace tooltips actionable (#11700) 2024-01-19 15:17:02 +01:00
Mathias Fredriksson 200a87e7d4 feat(cli/ssh): allow multiple remote forwards and allow missing local file (#11648) 2024-01-19 15:21:10 +02:00
Mathias Fredriksson 73e6bbff7e feat(cli/exp): add app testing to scaletest workspace-traffic (#11633) 2024-01-19 15:20:19 +02:00
Bruno Quaresma 1f63a11396 refactor(site): refactor resource and agents (#11647) 2024-01-19 09:06:33 -03:00
Marcin Tojek 89fd29478d feat: expose support links as env variables (#11697) 2024-01-19 11:20:36 +01:00
Garrett Delfosse bf0a6fcc32 feat: manage provisioner tags in template editor (#11600) 2024-01-18 17:35:20 -05:00
Kayla Washburn-Love 9ed3487f67 feat: batch workspace updates (#11583) 2024-01-18 15:14:25 -07:00
Bruno Quaresma 156aaba335 feat(site): show version files diff based on active version (#11686) 2024-01-18 16:08:17 -03:00
Steven Masley 6bb1a34a37 fix: allow ports in wildcard url configuration (#11657)
* fix: allow ports in wildcard url configuration

This just forwards the port to the ui that generates urls.
Our existing parsing + regex already supported ports for
subdomain app requests.
2024-01-18 09:44:05 -06:00
Spike Curtis 1f0e6ba6c6 fix: use raw syscalls to write binary we execute (#11684)
Fixes flake seen here, I think

https://github.com/coder/coder/actions/runs/7565915337/job/20602500818

golang's file processing is complex, and in at least some cases it can return from a file.Close() call without having actually closed the file descriptor.

If we're holding open the file descriptor of an executable we just wrote, and try to execute it, it will fail with "text file busy" which is what we have seen.

So, to be extra sure, I've avoided the standard library and directly called the syscalls to open, write, and close the file we intend to use in the test.

I've also added some more logging so if it's some issue of multiple tests writing to the same location, the we might have a chance to see it.
2024-01-18 16:21:11 +04:00
Marcin Tojek c5d73b86d6 feat: change owner name using account form (#11683) 2024-01-18 12:32:01 +01:00
Muhammad Atif Ali 1ea70ba573 ci: build a multi-arch image on each commit to main (#11544) 2024-01-18 10:57:35 +00:00
Spike Curtis 8910ac715c feat: add tailnet v2 support to wsproxy coordinate endpoint (#11637)
wsproxy also needs to be updated to use tailnet v2 because the `tailnet.Conn` stores peers by ID, and the peerID was not being carried by the JSON protocol.  This adds a query param to the endpoint to conditionally switch to the new protocol.
2024-01-18 10:10:36 +04:00
Spike Curtis 07427e06f7 chore: add setBlockEndpoints to nodeUpdater (#11636)
nodeUpdater also needs block endpoints, so that it can stop sending nodes with endpoints.
2024-01-18 10:02:15 +04:00
Spike Curtis 5b4de667d6 chore: add setCallback to nodeUpdater (#11635)
we need to be able to (re-)set the node callback when we lose and regain connection to a coordinator over the network.
2024-01-18 09:51:09 +04:00
Spike Curtis e725f9d7d4 chore: stop passing addresses on configMaps constructor (#11634)
moving this out of the constructor so that setting this when creating a new `tailnet.Conn` triggers configuring the engine.
2024-01-18 09:43:28 +04:00
Spike Curtis a514df71ed chore: add setDERPMap to configMaps (#11590)
Add setDERPMap
2024-01-18 09:34:30 +04:00
Spike Curtis 25e289e1f6 chore: add setAddresses to nodeUpdater (#11571)
Adds setAddresses to nodeUpdater
2024-01-18 09:24:16 +04:00
Spike Curtis 387723a596 fix: close pg PubSub listener to avoid race (#11640)
Fixes flake as seen here: https://github.com/coder/coder/runs/20528529187
2024-01-18 09:18:59 +04:00
Asher 72d9ec07aa fix: detect JetBrains running on local ipv6 (#11676) 2024-01-17 14:08:15 -09:00
Jon Ayers 552e9fe22f fix: avoid returning 500 on apps when workspace stopped (#11656) 2024-01-17 12:06:59 -06:00
Bruno Quaresma 1be119b08f fix(site): fix search menu for creating workspace and templates filter (#11674) 2024-01-17 17:54:56 +00:00
Steven Masley b246f08d84 chore: move app URL parsing to its own package (#11651)
* chore: move app url parsing to it's own package
2024-01-17 10:41:42 -06:00
Bruno Quaresma 1aee8da4b6 fix(site): fix sidebar scroll (#11671) 2024-01-17 16:05:05 +00:00
dependabot[bot] fa6176c2ff chore: bump github.com/u-root/u-root from 0.11.0 to 0.12.0 (#11625)
* chore: bump github.com/u-root/u-root from 0.11.0 to 0.12.0

Bumps [github.com/u-root/u-root](https://github.com/u-root/u-root) from 0.11.0 to 0.12.0.
- [Release notes](https://github.com/u-root/u-root/releases)
- [Changelog](https://github.com/u-root/u-root/blob/main/RELEASES)
- [Commits](https://github.com/u-root/u-root/compare/v0.11.0...v0.12.0)

---
updated-dependencies:
- dependency-name: github.com/u-root/u-root
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* `go mod tidy`

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Muhammad Atif Ali <me@matifali.dev>
Co-authored-by: Muhammad Atif Ali <atif@coder.com>
2024-01-18 01:36:47 +10:00
Marcin Tojek e83f13d8c5 fix: typo in whitespace (#11667) 2024-01-17 12:36:15 +00:00
Marcin Tojek 5eb3e1cdaa feat: expose owner_name in coder_workspace resource (#11639) 2024-01-17 13:20:45 +01:00
Spike Curtis b173195e0d Revert "fix: detect JetBrains running on local ipv6 (#11653)" (#11664)
This reverts commit 2d61d5332a.
2024-01-17 15:38:39 +04:00
Spike Curtis 2aa3cbbd03 chore: add logging to nodeUpdater (#11569)
Add debug logging for nodeUpdater and configMaps
2024-01-17 14:15:45 +04:00
Spike Curtis bad2ce562e fix: stop asserting fuzz bytes written in test
Fixes a flake seen here: https://github.com/coder/coder/actions/runs/7541558190/job/20528545916

```
=== FAIL: enterprise/provisionerd TestRemoteConnector_Fuzz (0.06s)
    t.go:84: 2024-01-16 12:32:27.024 [info]  connector: failed provisioner authentication  remote_addr=[::1]:45138 ...
        error= failed to receive jobID:
                   github.com/coder/coder/v2/enterprise/provisionerd.(*remoteConnector).authenticate
                       /home/runner/actions-runner/_work/coder/coder/enterprise/provisionerd/remoteprovisioners.go:438
                 - bufio.Scanner: token too long
    t.go:84: 2024-01-16 12:32:27.024 [debu]  connector: closed connection  remote_addr=[::1]:45138  error=<nil>
    remoteprovisioners_test.go:209: 
            Error Trace:    /home/runner/actions-runner/_work/coder/coder/enterprise/provisionerd/remoteprovisioners_test.go:209
            Error:          "2992256" is not less than "2097152"
            Test:           TestRemoteConnector_Fuzz
            Messages:       should not allow more than 1 MiB
```

This was an attempt to test that malicious actors can't abuse our authentication protocol to make us allocate a bunch of memory.
However, the test asserted on the number of bytes sent by the fuzzer, not the number of bytes read (& allocated) by the service.  The former is affected by network queue sizes and is thus flaky without actively managing the socket queues, which I don't think we want to do.

In actual practise, the thing that matters is how much memory the bufio Scanner allocates. By inspection, the scanner will allocate up to 64k, and testing this is true devolves into testing the go standard library, which I don't think is worth doing.

So... let's just drop the assertion because 

a) its flaky, 

b) it doesn't test what we actually want to test, 

c) the behavior we actually care about is part of the standard library.
2024-01-17 12:59:45 +04:00
Spike Curtis 38d9ce5267 chore: add setStatus support to nodeUpdater (#11568)
Add support for the wgengine Status callback to nodeUpdater
2024-01-17 09:06:34 +04:00
Spike Curtis f6dc707511 chore: add DERPForcedWebsocket to nodeUpdater (#11567)
Add support for DERPForcedWebsocket to nodeUpdater
2024-01-17 08:55:45 +04:00
Asher 2d61d5332a fix: detect JetBrains running on local ipv6 (#11653) 2024-01-16 15:53:41 -09:00
Colin Adler be43d6247d feat: add additional fields to first time setup trial flow (#11533)
* feat: add additional fields to first time setup trial flow

* trial generator typo
2024-01-16 18:19:16 -06:00
Jon Ayers 1196f83ebd feat: automatically activate dormant workspaces when manually started (#11655) 2024-01-16 16:42:04 -06:00
Stephen Kirby d74aae7a4a removed alpha tags from workspace actions features in template settings (#11654) 2024-01-16 16:23:19 -06:00
Muhammad Atif Ali 417270a6d7 chore(docs): remove the template_update_policies experiment from docs (#11615) 2024-01-17 00:18:57 +03:00
Jon Ayers 6ebcee3b49 docs: add workspace cleanup docs (#11146)
Co-authored-by: Muhammad Atif Ali <atif@coder.com>
Co-authored-by: kirby <kirby@coder.com>
2024-01-16 15:12:56 -06:00
Mathias Fredriksson 385d58caf6 fix(agent/agentssh): allow remote forwarding a socket multiple times (#11631)
* fix(agent/agentssh): allow remote forwarding a socket multiple times

Fixes #11198
Fixes https://github.com/coder/customers/issues/407
2024-01-16 21:26:13 +02:00
Steven Masley 08b4eb3124 fix: refresh all oauth links on external auth page (#11646)
* fix: refresh all oauth links on external auth page
2024-01-16 11:03:55 -06:00
Cian Johnston d583acad00 fix(coderd): workspaceapps: update last_used_at when workspace app reports stats (#11603)
- Adds a new query BatchUpdateLastUsedAt
- Adds calls to BatchUpdateLastUsedAt in app stats handler upon flush
- Passes a stats flush channel to apptest setup scaffolding and updates unit tests to assert modifications to LastUsedAt.
2024-01-16 14:06:39 +00:00
Muhammad Atif Ali 5bfbf9f9e6 chore(docs/install/docker.md): shorten headings length (#11630) 2024-01-16 07:19:58 +00:00
Steven Masley 5087f7b5f6 chore: improve fake IDP script (#11602)
* chore: testIDP using static defaults for easier reuse
2024-01-15 10:01:41 -06:00
Marcin Tojek f915bdf26c feat: support links with custom icons (#11629) 2024-01-15 16:56:01 +01:00
dependabot[bot] 5c310ec334 chore: bump github.com/prometheus/common from 0.45.0 to 0.46.0 (#11618)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-15 18:19:16 +03:00
dependabot[bot] 288f879f72 ci: bump the github-actions group with 1 update (#11616)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-15 14:48:19 +00:00
dependabot[bot] af013fc3a1 chore: bump github.com/go-playground/validator/v10 from 10.16.0 to 10.17.0 (#11626)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-15 17:45:49 +03:00
dependabot[bot] 476d72e63d chore: bump github.com/andybalholm/brotli from 1.0.6 to 1.1.0 (#11621)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-15 17:44:48 +03:00
dependabot[bot] ecefb8c0c1 chore: bump golang.org/x/tools from 0.16.1 to 0.17.0 (#11622)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-15 17:44:17 +03:00
Cian Johnston 244ca88645 ci: set CODER_VERBOSE=true for fly.io wsproxies (#11405) 2024-01-15 13:14:38 +00:00
dependabot[bot] 054420bb33 chore: bump github.com/go-logr/logr from 1.3.0 to 1.4.1 (#11475)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-15 14:08:28 +03:00
sharkymark f65b2efb95 chore: replace remote with cloud when referencing development environments; add Slack as an enterprise option for community sharing (#11375) 2024-01-15 05:31:35 -05:00
Eric Paulsen c799f0ff43 docs: add steps to configure supportLinks in Helm chart (#11612) 2024-01-15 05:29:59 -05:00
Eric Paulsen e1493b220a fix: guide naming (#11613) 2024-01-15 05:29:43 -05:00
Muhammad Atif Ali 8b10d21a70 chore(docs): fix a minor punctuation error (#11610) 2024-01-14 08:03:07 +00:00
Eric Paulsen e70a97a722 docs: add guide for template ImagePullSecret (#11608)
* docs: add guide for template imagepullsecret

* add: manifest

* make: fmt
2024-01-12 18:44:26 -06:00
Kayla Washburn-Love 4c3f05b8aa fix: show error when creating a new group fails (#11560) 2024-01-12 16:06:02 -07:00
Steven Masley 905292053a fix: improve wsproxy error when proxyurl is set to a primary (#11586)
* coder error first
2024-01-12 20:32:02 +00:00
Steven Masley 03ee63931c chore: remove duplicate validate calls on same oauth token (#11598)
* chore: remove duplicate validate calls on same oauth token
2024-01-12 14:27:22 -06:00
Bruno Quaresma 8181c9f349 refactor(site): make cosmetic changes on agent logs (#11601) 2024-01-12 17:09:36 -03:00
Bruno Quaresma 68e5a51d90 feat(site): display builds logs by default (#11597) 2024-01-12 16:39:23 -03:00
Bruno Quaresma ec166cf423 fix(site): remove search menu vertical padding (#11599) 2024-01-12 19:33:21 +00:00
Bruno Quaresma f3edc42b76 fix(site): fix workspace resource width on ultra wide screens (#11596) 2024-01-12 16:09:12 -03:00
Bruno Quaresma 130d5d68a0 refactor(site): refactor workspace notifications (#11520) 2024-01-12 15:55:31 -03:00
Stephen Kirby bdefd4e2e6 chore: convert faq headers to dropdowns (#11585)
* changed FAQs from headers to twists

* added dropdowns and mild formatting

* make fmt
2024-01-12 12:49:41 -06:00
Bruno Quaresma 162c91ec2a fix(site): fix resource selection when workspace has no prev resources (#11594) 2024-01-12 15:45:06 -03:00
Marcin Tojek cb77f04104 feat: load variables from tfvars files (#11549) 2024-01-12 15:08:23 +01:00
Bruno Quaresma aeb1ab8ad8 fix(site): fix resource selection when workspace resources change (#11581) 2024-01-12 10:14:31 -03:00
Cian Johnston 0e96115d5d fix(coderd): correctly show warning when no provisioner daemons are registered (#11591) 2024-01-12 11:22:59 +00:00
Steven Masley f5a9f5ca3d chore: handle errors in wsproxy server for cli using buildinfo (#11584)
Cli errors are pretty formatted. This handles nested pretty types. Before it found the first error it could understand and return that. Now it will print the full error stack with more information.

To prevent information loss, a "[Trace=...]" was added to capture some extra error context for debugging.
2024-01-11 16:55:34 -06:00
Jon Ayers aecdafdcf2 fix: fix template edit overriding with flag defaults (#11564) 2024-01-11 16:18:46 -06:00
Kayla Washburn-Love eb8d85f432 feat: treat deprecation messages as markdown (#11562) 2024-01-11 14:15:29 -07:00
Cian Johnston 95fd0bb22b feat(site): remove experiment deployment_health_page (#11572) 2024-01-11 21:03:10 +00:00
Cian Johnston 26f5ce63a8 feat(site): add docs links on health page (#11582)
* feat(site): add docs links on health page

* apply suggestions
2024-01-11 20:32:25 +00:00
Garrett Delfosse 5b122d108e fix: publish workspace update on quota failure (#11559) 2024-01-11 14:59:40 -05:00
Kayla Washburn-Love 05eac64be4 feat: add a character counter for fields with length limits (#11558)
- refactors`getFormHelpers` to accept an options object
- adds a `maxLength` option which will display a message and character counter for fields with length limits
- set `maxLength` option for template description fields
2024-01-11 12:15:43 -07:00
Garrett Delfosse f9f94b5d01 fix: remove cancel button if user cannot cancel job (#11553) 2024-01-11 13:48:44 -05:00
Kayla Washburn-Love 8c3a4f2d7f chore: move some components into pages/ (#11536) 2024-01-11 11:30:15 -07:00
Steven Masley e3ad9580e9 chore: allow running fake idp with coderd dev (#11555)
* chore: allow running fake idp with coderd dev
2024-01-11 18:10:57 +00:00
sharkymark c91b885a4a chore: add optional coder_app to faq (#11351)
Merging since Mark is out.

* chore: add optional coder_app to faq

* applied Atif's suggestions

* make fmt again

---------

Co-authored-by: kirby <kirby@coder.com>
Co-authored-by: Stephen Kirby <58410745+stirby@users.noreply.github.com>
2024-01-11 12:07:22 -06:00
Steven Masley fcd299109c chore: update language about autostop on templates page (#11552)
* chore: update language about autostop on templates page
2024-01-11 12:01:07 -06:00
Steven Masley 8b61ff3e0e fix: apply appropriate artifactory defaults for external auth (#11580) 2024-01-11 11:58:27 -06:00
Cian Johnston f3d091fa01 fix(site): improve rendering of provisioner tags (#11575)
* fix(site): improve rendering of provisioner tags

* fixup! fix(site): improve rendering of provisioner tags

* Update site/src/pages/HealthPage/ProvisionerDaemonsPage.tsx

* fixup! Update site/src/pages/HealthPage/ProvisionerDaemonsPage.tsx
2024-01-11 17:42:21 +00:00
Colin Adler 4a0808259a fix: ensure wsproxy MultiAgent is closed when websocket dies (#11414)
The `SingleTailnet` behavior only checked to see if the `MultiAgent` was
closed, but the websocket error was not being propogated into the
`MultiAgent`, causing it to never be swapped for a new working one.

Fixes https://github.com/coder/coder/issues/11401

Before:
```
Coder Workspace Proxy v0.0.0-devel+85ff030 - Your Self-Hosted Remote Development Platform
Started HTTP listener at http://0.0.0.0:3001

View the Web UI: http://127.0.0.1:3001

==> Logs will stream in below (press ctrl+c to gracefully exit):
2024-01-04 20:11:56.376 [warn]  net.workspace-proxy.servertailnet: broadcast server node to agents ...
    error= write message:
               github.com/coder/coder/v2/enterprise/wsproxy/wsproxysdk.(*remoteMultiAgentHandler).writeJSON
                   /home/coder/coder/enterprise/wsproxy/wsproxysdk/wsproxysdk.go:524
             - failed to write msg: WebSocket closed: failed to read frame header: EOF
```

After:
```
Coder Workspace Proxy v0.0.0-devel+12f1878 - Your Self-Hosted Remote Development Platform
Started HTTP listener at http://0.0.0.0:3001

View the Web UI: http://127.0.0.1:3001

==> Logs will stream in below (press ctrl+c to gracefully exit):
2024-01-04 20:26:38.545 [warn]  net.workspace-proxy.servertailnet: multiagent closed, reinitializing
2024-01-04 20:26:38.546 [erro]  net.workspace-proxy.servertailnet: reinit multi agent ...
    error= dial coordinate websocket:
               github.com/coder/coder/v2/enterprise/wsproxy/wsproxysdk.(*Client).DialCoordinator
                   /home/coder/coder/enterprise/wsproxy/wsproxysdk/wsproxysdk.go:454
             - failed to WebSocket dial: failed to send handshake request: Get "http://127.0.0.1:3000/api/v2/workspaceproxies/me/coordinate": dial tcp 127.0.0.1:3000: connect: connection refused
2024-01-04 20:26:38.587 [erro]  net.workspace-proxy.servertailnet: reinit multi agent ...
    error= dial coordinate websocket:
               github.com/coder/coder/v2/enterprise/wsproxy/wsproxysdk.(*Client).DialCoordinator
                   /home/coder/coder/enterprise/wsproxy/wsproxysdk/wsproxysdk.go:454
             - failed to WebSocket dial: failed to send handshake request: Get "http://127.0.0.1:3000/api/v2/workspaceproxies/me/coordinate": dial tcp 127.0.0.1:3000: connect: connection refusedhandshake request: Get "http://127.0.0.1:3000/api/v2/workspaceproxies/me/coordinate": dial tcp 127.0.0.1:3000: connect: connection refused
2024-01-04 20:26:40.446 [info]  net.workspace-proxy.servertailnet: successfully reinitialized multiagent  agents=0  took=1.900892615s
```
2024-01-11 11:37:09 -06:00
Bruno Quaresma d708ac7c04 fix(site): remove refetch on windows focus (#11574)
It causes the sign-in page to reload whenever a user enters a page or changes the window's focus. This is happening because when the "user" fetch is made, the server returns an error, making the react-query mark the data as stale and try to load it whenever possible.
2024-01-11 11:06:36 -03:00
Bruno Quaresma 3695b74ab6 fix(site): fix loading indicator alignment (#11573) 2024-01-11 10:53:36 -03:00
Cian Johnston 8a12ee7831 fix(site): show wsproxy errors in context in WorkspaceProxyPage (#11556)
* Shows the overall report error at the top of the page, if present.
* Shows workspaceproxy errors above warnings inside the corresponding element, if present.
* Improves unregistered proxy status
2024-01-11 10:47:02 +00:00
Spike Curtis 8701dbc874 chore: add nodeUpdater to tailnet (#11539)
Adds a nodeUpdater component, which serves a similar role to configMaps, but tracks information from tailscale going out to the coordinator as node updates.  This first PR just handles netInfo, subsequent PRs will
handle DERP forced websockets, endpoints, and addresses.
2024-01-11 09:29:42 +04:00
Spike Curtis 7005fb1b2f chore: add support for blockEndpoints to configMaps (#11512)
Adds support for setting blockEndpoints on the configMaps
2024-01-11 09:18:31 +04:00
Spike Curtis 617ecbfb1f chore: add support for peer updates to tailnet.configMaps (#11487)
Adds support to configMaps to handle peer updates including lost and disconnected peers
2024-01-11 09:11:43 +04:00
bamhm182 4e5367c4a4 chore: update Digital Ocean example template (#11528) (#11535)
Co-authored-by: Muhammad Atif Ali <atif@coder.com>
2024-01-11 00:00:25 +00:00
Jon Ayers 9b437032e9 feat: remove workspace_actions experiment (#11427) 2024-01-10 16:19:39 -06:00
Steven Masley 04afb88e6f fix: return a more sophisticated error for device failure on 429 (#11554)
* fix: return a more sophisticated error for device failure on 429
2024-01-10 11:29:44 -06:00
Mathias Fredriksson b1d53a68c2 fix(agent/agentssh): fix X11 forwarding by improving Xauthority management (#11550)
Fixes #11531
2024-01-10 19:04:44 +02:00
Steven Masley 89ab659114 chore: add oauth2 prometheus metrics for to documentation (#11534) 2024-01-10 15:46:37 +00:00
Steven Masley 3f9da674c6 chore: instrument github oauth2 limits (#11532)
* chore: instrument github oauth2 limits

Rate limit information for github oauth2 providers instrumented in prometheus
2024-01-10 15:29:33 +00:00
Steven Masley 50b78e3325 chore: instrument external oauth2 requests (#11519)
* chore: instrument external oauth2 requests

External requests made by oauth2 configs are now instrumented into prometheus metrics.
2024-01-10 09:13:30 -06:00
Garrett Delfosse aa7fe075a8 fix: correct flag name (#11525) 2024-01-10 09:36:26 -05:00
Garrett Delfosse 0727535342 fix: correct app url format in comment (#11523) 2024-01-10 09:36:10 -05:00
Muhammad Atif Ali 6e5c2efca1 chore(docs): remove provider logos from 1-click install (#11548)
* docs: remove cloud logos from 1-click install

They were looking good and are not adding much value.

* Delete docs/images/install/render.png

* Delete docs/images/install/ec2.svg

* Delete docs/images/install/eks.svg

* Delete docs/images/install/fly.io.svg

* Delete docs/images/install/gce.svg

* Delete docs/images/install/heroku.svg

* Delete docs/images/install/railway.svg
2024-01-10 13:28:40 +00:00
Spike Curtis cae095fdb6 fix: stop logging errors on canceled cleanup queries (#11547)
Fixes flake seen here: https://github.com/coder/coder/actions/runs/7474259128/job/20340051975
2024-01-10 16:20:29 +04:00
Muhammad Atif Ali 9682db593e chore(docs): reorganize installation docs (#11465) 2024-01-10 15:00:19 +03:00
Spike Curtis dfe8efc186 fix: use background context for inmem provisionerd (#11545)
This test case fails with an error log, showing "context canceled" when trying to send an acquired job to an in-mem provisionerd.

https://github.com/coder/coder/runs/20331469006

In this case, we don't want to supress this error, since it could mean that we acquired a job, locked it in the database, then failed to send it to a provisioner.
(We also don't want to mark the job as failed because we don't know whether the job made it to the provisionerd or not --- in the failed test you can see that the job is actually processed just fine).

The reason we got context canceled is because the API was shutting down --- we don't want provisionerdserver to abruptly stop processing job stuff as the API shuts down as this will leave jobs in a bad state.  This PR fixes up the use of contexts with provisionerdserver and the associated drpc service calls.
2024-01-10 15:29:57 +04:00
Muhammad Atif Ali c125206b24 docs(faqs): add FAQ regarding unsupported base image for VS Code Server (#11543) 2024-01-10 12:16:44 +03:00
Cian Johnston 5ecb0db4f2 chore(coderd): fix test flake in TestAgentWebsocketMonitor_SendPings (#11518) 2024-01-10 08:45:46 +00:00
Cian Johnston 5ed3c413cd chore(coderd): fix test flake in TestWorkspaceUpdateAutomaticUpdates_OK (#11521) 2024-01-10 08:45:32 +00:00
dependabot[bot] 61cd9f087b chore: bump follow-redirects from 1.15.2 to 1.15.4 in /site (#11540)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-10 10:30:05 +03:00
Spike Curtis 89e3bbe0f5 chore: add configMaps component to tailnet (#11400)
Work in progress on a subcomponent of the Conn which will handle configuring the wireguard engine on changes.  I've implemented setAddresses as the simplest case and added unit tests of the reconfiguration loop.

Besides making the code easier to test and understand, the goal is for this component to handle disconnect and loss updates about peers, and thereby, implement the v2 Tailnet API.

Further PRs will handle peer updates, status updates, and net info updates.

Then, after the subcomponent is implemented and tested, I will refactor Conn to use it instead of the current monolithic architecture.
2024-01-10 10:58:53 +04:00
Asher d837d66e29 chore: update sqlc to 1.25.0 (#11538)
Co-authored-by: Muhammad Atif Ali <atif@coder.com>
2024-01-10 09:19:41 +03:00
Asher 0912cfc2d6 chore: update flake to include new mockgen (#11537)
It looks like we updated mockgen to use Uber's fork, but the flake still
pointed to a nixos-unstable commit containing the old mockgen resulting 
in an error like:

missing go.sum entry for module providing package github.com/golang/mock/mockgen/model
2024-01-09 15:37:51 -09:00
Kayla Washburn 97bd74b468 chore: add additional stories to storybook (#11524)
add stories for ActiveUserChart, CopyableValue, and CopyButton
2024-01-09 14:03:40 -07:00
Kayla Washburn 8a48485014 refactor: clean up Welcome component (#11526) 2024-01-09 14:03:33 -07:00
Garrett Delfosse 4fa07124cd feat: display application name over sign in form (#11500) 2024-01-09 12:51:16 -05:00
Garrett Delfosse 30d5ac060b fix: carry tags to new templateversions (#11502) 2024-01-09 12:47:44 -05:00
Cian Johnston 952706e905 fix(site): HealthPage/WorkspaceProxyPage: adjust border colour for unhealthy regions (#11516) 2024-01-09 17:36:41 +00:00
Kayla Washburn e77b1a5ffd chore: miscellaneous cleanup (#11027) 2024-01-09 10:14:19 -07:00
Cian Johnston 9f4f953350 fix(coderd/healthcheck): ignore deleted wsproxies in wsproxy healthcheck (#11515) 2024-01-09 16:36:26 +00:00
Marcin Tojek e5b9d63901 docs: escape enum pipe (#11513) 2024-01-09 13:39:38 +00:00
Marcin Tojek 525e6e5dc8 docs: remove empty page (#11511) 2024-01-09 12:52:45 +01:00
Marcin Tojek b8373e6fab fix: nix: force node version v18 (#11510) 2024-01-09 12:27:56 +01:00
Spike Curtis fdd60d316e fix: fix MetricsAggregator check for metric sameness (#11508)
Fixes #11451

A refactor of the Agent API passes metrics as protobufs, which include pointers to label name/value pairs.  The aggregator tested for sameness by doing a shallow compare of label values, which for different stats reports would compare unequal because the pointers would be different.

This fix does a deep compare.

While testing I also noted that we neglect to compare template names. This is unlikely to have caused any issue in practice, since the combination of username/workspace is unique, but in the context of comparing metric labels we should do the comparison.

If a user creates a workspace, deletes it, then recreates from a different template, we could in principle have reported incorrect stats for the old template.
2024-01-09 15:21:30 +04:00
Spike Curtis 21093c00f0 fix: stop logging error on canceled query (#11506)
Fixes flake seen here: https://github.com/coder/coder/actions/runs/7447779208/job/20260756050
2024-01-09 14:38:56 +04:00
Cian Johnston 0c953b4b8c fix(enterprise/coderd): make primary workspace proxy always be updatd now (#11499) 2024-01-09 10:03:08 +00:00
Steven Masley fb29af664b fix: relax csrf to exclude path based apps (#11430)
* fix: relax csrf to exclude path based apps
* add unit test to verify path based apps are not CSRF blocked
2024-01-08 22:33:57 +00:00
Kayla Washburn 9f5a59d5c5 feat(site): improve icon compatibility across themes (#11457) 2024-01-08 14:12:40 -07:00
Garrett Delfosse 427afe13e0 fix: generate new random username to prevent flake (#11501) 2024-01-08 19:09:14 +00:00
Cian Johnston 220e95dd5c feat(site): add healthcheck page for provisioner daemons (#11494)
Part of #10676

- Adds a health section for provisioner daemons (mostly cannibalized from the Workspace Proxy section)
- Adds a corresponding storybook entry for provisioner daemons health section
- Fixed an issue where dismissing the provisioner daemons warnings would result in a 500 error
- Adds provisioner daemon error codes to docs
2024-01-08 17:14:09 +00:00
dependabot[bot] 6096af77c8 chore: bump github.com/cloudflare/circl from 1.3.3 to 1.3.7 (#11495)
Bumps [github.com/cloudflare/circl](https://github.com/cloudflare/circl) from 1.3.3 to 1.3.7.
- [Release notes](https://github.com/cloudflare/circl/releases)
- [Commits](https://github.com/cloudflare/circl/compare/v1.3.3...v1.3.7)

---
updated-dependencies:
- dependency-name: github.com/cloudflare/circl
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 20:10:59 +03:00
Bruno Quaresma a613a0a4da refactor(site): improve settings option (#11489) 2024-01-08 13:16:16 -03:00
Bruno Quaresma 61450863ff feat(site): move resources into the sidebar (#11456) 2024-01-08 13:14:25 -03:00
dependabot[bot] 359a642e7e chore: bump github.com/prometheus/client_golang from 1.17.0 to 1.18.0 (#11474)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 17:07:56 +03:00
Cian Johnston 93cf5dcd47 fix(coderd/healthcheck): add daemon-specific warnings to healthcheck output (#11490)
- Sorts provisioner daemons by name ascending in output
- Adds daemon-specific warnings to healthcheck output
- Reword some messages
2024-01-08 13:55:00 +00:00
dependabot[bot] f4393d0c3f chore: bump github.com/hashicorp/terraform-json from 0.18.0 to 0.20.0 (#11483)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 13:20:14 +00:00
Bruno Quaresma 0186241880 fix(site): display github login config (#11488) 2024-01-08 10:17:09 -03:00
dependabot[bot] efb1ee31c0 chore: bump github.com/unrolled/secure from 1.13.0 to 1.14.0 (#11476)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 16:15:53 +03:00
dependabot[bot] 4c7a93dd7e chore: bump github.com/coreos/go-oidc/v3 from 3.7.0 to 3.9.0 (#11479)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 15:32:05 +03:00
dependabot[bot] a6c746e4e0 chore: bump github.com/aws/smithy-go from 1.17.0 to 1.19.0 (#11484)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 15:31:43 +03:00
dependabot[bot] 2c9589d883 chore: bump github.com/google/uuid from 1.4.0 to 1.5.0 (#11485)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 15:31:23 +03:00
dependabot[bot] 58f5f324b0 chore: bump github.com/gohugoio/hugo from 0.120.3 to 0.121.2 (#11473)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 15:31:09 +03:00
Muhammad Atif Ali acec1f7716 chore: increase dependabot PRs limit for go (#11472) 2024-01-08 14:54:28 +03:00
dependabot[bot] 5337a70561 chore: bump google.golang.org/protobuf from 1.31.0 to 1.32.0 (#11468)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 13:43:33 +03:00
Cian Johnston 04fd96a014 feat(coderd): add provisioner_daemons to /debug/health endpoint (#11393)
Adds a healthcheck for provisioner daemons to /debug/health endpoint.
2024-01-08 09:29:04 +00:00
Michael Smith 31f8fac1b9 fix: make ProxyMenu more accessible to screen readers (#11312)
* wip: commit progress on latency update

* chore: add stories and clean up tests

* refactor: clean up code

* fix: make sure headers aren't treated as interactive elements

* refactor: clean up tests

* fix: clean up stories

* docs: add clarifying comment

* fix: update stories again

* fix: clean up/extend prop definitions

* refactor: quick cleanup

* fix: apply Kira's feedback

* refactor: clean up abbr markup to account for pronunciation

* fix: more cleanup

* fix: refine screen reader output for VoiceOver

* refactor: clean up and redefine tests

* feature: add finishing touches
2024-01-07 18:37:01 -05:00
dependabot[bot] 8a9fe2bf00 chore: bump golang.org/x/term from 0.15.0 to 0.16.0 (#11463)
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.15.0 to 0.16.0.
- [Commits](https://github.com/golang/term/compare/v0.15.0...v0.16.0)

---
updated-dependencies:
- dependency-name: golang.org/x/term
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-07 12:28:39 +00:00
dependabot[bot] 64f239c844 chore: bump github.com/go-chi/httprate from 0.7.4 to 0.8.0 (#11461)
Bumps [github.com/go-chi/httprate](https://github.com/go-chi/httprate) from 0.7.4 to 0.8.0.
- [Commits](https://github.com/go-chi/httprate/compare/v0.7.4...v0.8.0)

---
updated-dependencies:
- dependency-name: github.com/go-chi/httprate
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-07 12:27:29 +00:00
dependabot[bot] ceb0ec43ad chore: bump google.golang.org/grpc from 1.59.0 to 1.60.1 (#11444)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.59.0 to 1.60.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.59.0...v1.60.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-07 12:18:00 +00:00
Muhammad Atif Ali a2f86e5e5e chore(dogfood): install corepack (#11459) 2024-01-06 03:11:16 +03:00
Muhammad Atif Ali 965b1e69e2 ci: add variable to template push in dogfood.yaml (#11458) 2024-01-05 21:35:49 +00:00
Garrett Delfosse b21da38bea chore: deprecate template create command in favor of template push (#11390) 2024-01-05 21:04:14 +00:00
Garrett Delfosse 3d54bc06f6 feat: display current version on coder list (#11450)
* feat: display current version on coder list

* fix make gen

* update golden
2024-01-05 15:33:08 -05:00
Muhammad Atif Ali 31f7b39513 chore(dogfood): update dogfood template to use artifactory (#11452)
* chore(dogfood): update to use artifactory

* Update main.tf
2024-01-05 23:25:51 +03:00
Steven Masley da7859c445 chore: change language on autostop (#11454)
* chore: change language on autostop
2024-01-05 11:40:25 -06:00
Bruno Quaresma c428395d71 feat(site): move history into sidebar (#11413) 2024-01-05 13:32:05 -03:00
Steven Masley f0132b543d fix: fix workspace proxy command app link href (#11423)
* fix: workspace proxy command app link href
2024-01-05 10:27:06 -06:00
dependabot[bot] 46b90ce898 chore: bump github.com/golang-migrate/migrate/v4 from 4.16.0 to 4.17.0 (#11446)
Bumps [github.com/golang-migrate/migrate/v4](https://github.com/golang-migrate/migrate) from 4.16.0 to 4.17.0.
- [Release notes](https://github.com/golang-migrate/migrate/releases)
- [Changelog](https://github.com/golang-migrate/migrate/blob/master/.goreleaser.yml)
- [Commits](https://github.com/golang-migrate/migrate/compare/v4.16.0...v4.17.0)

---
updated-dependencies:
- dependency-name: github.com/golang-migrate/migrate/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-05 09:42:53 -06:00
dependabot[bot] f3efa0803b ci: bump the github-actions group with 3 updates (#11447)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-05 14:11:09 +00:00
dependabot[bot] 45e989a519 chore: bump golang.org/x/sync from 0.5.0 to 0.6.0 (#11445)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-05 17:09:11 +03:00
dependabot[bot] bf00e61f10 chore: bump github.com/jedib0t/go-pretty/v6 from 6.4.0 to 6.5.0 (#11442)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-05 16:51:19 +03:00
Muhammad Atif Ali 118ab7d4de ci: ungroup go dependencies (#11441) 2024-01-05 13:40:10 +00:00
Bruno Quaresma 9389c2b283 refactor(site): only show derp tags if they are true (#11439) 2024-01-05 09:45:34 -03:00
Cian Johnston 4d2fe2685a chore(coderd): extract api version validation to util package (#11407) 2024-01-05 10:22:07 +00:00
Spike Curtis 58873fa7e2 chore: remove unused context/cancel in tailnet Conn (#11399)
Spotted during code read; unused fields
2024-01-05 08:15:42 +04:00
Spike Curtis 64638b381d feat: promote PG Coordinator out of experimental (#11398)
Promotes PG Coordinator out of experimental to GA
2024-01-05 08:03:36 +04:00
Eric Paulsen e816dc0e60 fix: gcp federation guide formatting (#11432) 2024-01-05 03:31:05 +00:00
Eric Paulsen 138d31621f docs: add guide for Google to AWS federation (#11429)
* feat: add docs for Google to AWS federation

* make: fmt
2024-01-04 20:13:29 -05:00
Steven Masley dd05a6b13a chore: mockgen archived, moved to new location (#11415)
* chore: mockgen archived, moved to new location
2024-01-04 18:35:56 -06:00
dependabot[bot] bb3510631b chore: bump the offlinedocs group in /offlinedocs with 1 update (#11428)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-05 02:03:26 +03:00
Steven Masley c6366e5b73 chore: prevent nil derefs in non-critical paths (#11411)
* chore: prevent nil derefs in non-critical paths

---------

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2024-01-04 14:51:48 -06:00
Steven Masley 85ff030ab4 chore: update LastConnectedReplicaID in dbmem (#11412) 2024-01-04 19:18:54 +00:00
Cian Johnston 4f433e7f3d ci: broaden scope of needs.changes.db (#11386)
* Broadens scope of needs.changes.db to include anything under the path coderd/database.
* Removes dependency of test-go-pg on sqlc-vet.
2024-01-04 16:24:54 +00:00
Garrett Delfosse 5981abd689 fix: handle unescaped userinfo in postgres url (#11396)
* fix: handle unescaped userinfo in postgres url

* add tests

* fix tests
2024-01-04 08:46:00 -05:00
Muhammad Atif Ali f0db302df2 chore: add lxc logo (#11404) 2024-01-04 16:31:16 +03:00
Cian Johnston 4355894b2b fix(coderd/database): revert addition of v prefix to provisioner_daemons.api_version (#11403)
"Reverts" #11385 by adding an inverse migration.
2024-01-04 11:47:31 +00:00
Spike Curtis f9ebe8c719 fix: send end of logs when dbfake completes job (#11402) 2024-01-04 15:18:00 +04:00
Spike Curtis 48cd4c3a10 feat: promote single-tailnet out of experimental (#11366) 2024-01-04 09:27:36 +04:00
Kayla Washburn ffa7722c31 feat: select group avatars with the emoji picker (#11395) 2024-01-03 15:47:24 -07:00
Ben Potter ee2daedae0 chore: template update policies are GA (#11397) 2024-01-03 15:49:29 -06:00
Dean Sheather 06f519d7f1 docs: add template autostop requirement docs (#11235)
* chore: template autostop requirement docs

* fixup! chore: template autostop requirement docs

* fixes from feedback.

* fmt

---------

Co-authored-by: Ben <me@bpmct.net>
2024-01-03 14:25:25 -06:00
Kyle Carberry 30afe43f8a fix: create tempdir prior to cleanup (#11394)
See https://github.com/coder/coder/actions/runs/7399827933/job/20132407700

Seems like this happened because the test was being cleaned up
while the tempdir was being made.
2024-01-03 19:18:15 +00:00
Bruno Quaresma 4edd21ae9e fix(site): fix loading spinner on template version status badge (#11392) 2024-01-03 16:29:04 +00:00
Bruno Quaresma f5196c06e1 fix(site): fix insights picker and disable animation (#11391) 2024-01-03 16:25:01 +00:00
Cian Johnston f80a1cf3c8 fix(coderd/database): add missing v prefix to provisioner_daemons.api_version (#11385) 2024-01-03 14:11:02 +00:00
Cian Johnston 068e730046 chore(coderd/database/dbfake): fix pq test flake in TestStart_Starting (#11384) 2024-01-03 12:27:50 +00:00
Bruno Quaresma d74e7ca20f docs: update FE fetching data docs (#11376) 2024-01-03 12:27:33 +00:00
Spike Curtis 5d76210b0d fix: change coder start to be a no-op if workspace is started
Fixes #11380
2024-01-03 13:24:37 +04:00
Cian Johnston 1ef96022b0 feat(coderd): add provisioner build version and api_version on serve (#11369)
* assert provisioner daemon version and api_version in unit tests
* add build info in HTTP header, extract codersdk.BuildVersionHeader
* add api_version to codersdk.ProvisionerDaemon
* testutil.MustString -> testutil.MustRandString
2024-01-03 09:01:57 +00:00
Muhammad Atif Ali 9031b498ea ci: use depot.dev to build dogfood image (#11378) 2024-01-02 23:27:37 +03:00
Garrett Delfosse 227234ded5 fix: correct wording on logo url field (#11377) 2024-01-02 14:53:18 -05:00
Bruno Quaresma ac899be74c chore(site): move workspace topbar component (#11374) 2024-01-02 16:02:11 -03:00
Mathias Fredriksson df3c310379 feat(cli): add coder open vscode (#11191)
Fixes #7667
2024-01-02 20:46:18 +02:00
Bruno Quaresma 099be249a7 fix(site): fix external auth button loading state (#11373) 2024-01-02 18:30:05 +00:00
Kayla Washburn 6308a78365 chore: clean up light theme code (#11319) 2024-01-02 15:19:20 -03:00
Bruno Quaresma c37c0e7d1b refactor(site): simplify workspace topbar (#11370) 2024-01-02 15:17:42 -03:00
Bruno Quaresma 62a20e86fd chore(site): ignore deletion date on chromatic (#11372) 2024-01-02 18:17:24 +00:00
Bruno Quaresma a1341ee9ac fix(site): fix pill spinner size (#11368) 2024-01-02 15:05:20 -03:00
Bruno Quaresma 467a1a3e71 fix(site): fix workspace topbar back button (#11371) 2024-01-02 18:01:06 +00:00
Kayla Washburn a24c3b4dc7 chore: cleanup inline prop type definitions (#11317) 2024-01-02 10:39:00 -07:00
Bruno Quaresma cf17fabcc6 feat(site): refactor workspace header to be more slim (#11327) 2024-01-02 12:42:51 -03:00
Muhammad Atif Ali 608937c79c chore(site): update node to version 18.19.0 (#11344) 2024-01-02 12:41:24 +00:00
Bruno Quaresma 8717fdfc20 refactor(site): refactor pill component API (#11329)
Refactor the Pill API to make it easier to extend and reuse.
2024-01-02 09:28:51 -03:00
Spike Curtis c9b7d61769 chore: refactor agent connection updates (#11301)
Refactors the code that handles monitoring an agent websocket with pings and updating the connection times in the DB.

Consolidates v1 and v2 agent APIs under the same code for this.

One substantive change (not _just_ a refactor) is that I've made it so that we actually disconnect if the agent fails to respond to our pings, rather than the old behavior where we would update the database, but not actually tear down the websocket.
2024-01-02 16:04:37 +04:00
Spike Curtis 520c3a8ff7 fix: use TSMP for pings and checking reachability (#11306)
We're seeing some flaky tests related to agent connectivity - https://github.com/coder/coder/actions/runs/7286675441/job/19856270998

I'm pretty sure what happened in this one is that the client opened a connection while the wgengine was in the process of reconfiguring the wireguard device, so the fact that the peer became "active" as a result of traffic being sent was not noticed.

The test calls `AwaitReachable()` but this only tests the disco layer, so it doesn't wait for wireguard to come up.

I think we should be using TSMP for pinging and reachability, since this operates at the IP layer, and therefore requires that wireguard comes up before being successful.

This should also help with the problems we have seen where a TCP connection starts before wireguard is up and the initial round trip has to wait for the 5 second wireguard handshake retry.

fixes: #11294
2024-01-02 15:53:52 +04:00
Muhammad Atif Ali 58e40f6cd6 chore: update nfpm to v2.35.1 (#11310) 2024-01-02 10:27:46 +00:00
Spike Curtis 4071f1713b feat: add logging to agent stats and JetBrains tracking (#11364)
Adds logging so we can hope to diagnose #11363
2024-01-02 13:34:49 +04:00
dependabot[bot] 893a8ea583 chore: bump golang.org/x/tools from 0.15.0 to 0.16.1 (#11357)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-02 12:26:45 +03:00
dependabot[bot] a439507c6a ci: bump the github-actions group with 1 update (#11355)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-02 12:26:29 +03:00
Steven Masley 8f49f10134 chore: put overrides and renames in pkg context in sqlc.yaml (#11347)
* chore: Put overrides and renames in pkg context in sqlc.yaml

---------

Co-authored-by: Andrew Benton <andrewmbenton@gmail.com>
2024-01-02 08:56:38 +00:00
Spike Curtis 36636bb6a5 feat: add tailnet to agent RPC service (#11304)
Adds tailnet.DRPCService to the agent API

Supports #10531 but we still need to add version negotiation to the websocket endpoint
2024-01-02 10:10:20 +04:00
Spike Curtis 25f2abf9ab chore: remove tailnet from agent API and rename client API to tailnet (#11303)
Refactors our DRPC service definitions slightly.

In the previous version, I inserted the RPCs from the tailnet proto directly into the Agent service.  This makes things hard to deal with because DRPC then generates a new set of methods with new interfaces with the `DRPCAgent_` prefixed.  Since you can't have a single method that takes different argument types, we couldn't reuse the implementation of those RFCs without a lot of extra classes and pass-thru methods.

Instead, the "right" way to do it is to integrate at the DRPC layer.  So, we have two DRPC services available over the Agent websocket, and register them both on the DRPC `mux`.

Since the tailnet proto RPC service is now for both clients and agents, I renamed some things to clarify and shorten.

This PR also removes the `TailnetAPI` implementation from the `agentapi` package, and the next PR in the stack replaces it with the implementation from the `tailnet` package.
2024-01-02 10:02:45 +04:00
Spike Curtis 65290997c1 chore: disable failing metrics check until it can be fixed (#11361) 2024-01-02 05:39:48 +00:00
Spike Curtis f28f340c7b fix: test for expiry 3 months on Azure certs (#11362) 2024-01-02 09:30:36 +04:00
Spike Curtis d257f8163d feat: implement DERP streaming on tailnet Client API (#11302)
Implements DERPMap streaming from client API.

In a subsequent PR I plan to remove the implementation in coderd/agentapi in favor of the tailnet one
2024-01-02 08:07:57 +04:00
dependabot[bot] 055a160431 chore: bump the offlinedocs group in /offlinedocs with 1 update (#11358)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-02 05:49:33 +03:00
sharkymark 3582284977 chore: update images in appearance docs and correct inconsistencies (#11346)
* chore: update images in appearance docs and correct inconsistencies

* fix: spelling

---------

Co-authored-by: Eric <ericpaulsen@coder.com>
2023-12-28 15:06:54 -06:00
Myne 6fcc49f030 fix(examples/templates/nomad-docker): ignore NOMAD_NAMESPACE and NOMAD_REGION when Coder is running in nomad (#11341) 2023-12-28 10:24:18 +03:00
Muhammad Atif Ali e9437e2662 chore(site): update miscellaneous svg icons (#11343) 2023-12-27 23:05:03 +03:00
Muhammad Atif Ali e3a1bdb60d chore(dogfood): update nodejs installation method (#11339) 2023-12-26 16:53:41 +03:00
Muhammad Atif Ali 0ebd656cd1 refactor: refactor JFrog docs and template (#11336) 2023-12-25 07:26:34 +00:00
Muhammad Atif Ali 5a558b69c3 chore(examples/jfrog): always install the latest JFrog extension (#11335) 2023-12-24 13:59:04 +03:00
Muhammad Atif Ali b69ccab390 fix(docs): add missing scoped token resource to JFrog docs (#11334) 2023-12-24 13:30:52 +03:00
Muhammad Atif Ali ed3ecfc923 chore: build dogfood image on PRs and skip pushing to registry (#11311) 2023-12-24 11:43:38 +03:00
Muhammad Atif Ali efe8c67774 ci: fix close reason type for stale issues
The action was faking because we were incorrectly using `not planned` instead of `not_planned`.
2023-12-23 18:43:13 +03:00
Mathias Fredriksson be3889af07 test(site/e2e): catch missing agent defaults in fillResource (#11105) 2023-12-23 11:52:27 +00:00
Yonatan Arbel 8271cb01c0 docs: fix broken link to JFrog module (#11322) 2023-12-22 14:42:59 +03:00
Cian Johnston 19abde12fb chore(coderd): fix test flake with auditor (#11316) 2023-12-22 09:50:49 +00:00
Michael Smith 167c15238a fix: prevent UI from jumping around when selecting workspaces (#11321) 2023-12-21 22:36:42 +00:00
Ben Potter b3e3521274 docs: add v2.6.0 changelog (#11320)
* docs: add v2.6.0 changelog

* fmt
2023-12-21 22:33:13 +00:00
Kayla Washburn 029c92fede fix: fix name for external auth connections (#11318) 2023-12-21 15:27:16 -07:00
Kayla Washburn db71c0fa54 refactor: remove theme "color palettes" (#11314) 2023-12-21 14:45:54 -07:00
Asher 5cfa34b31e feat: add OAuth2 applications (#11197)
* Add database tables for OAuth2 applications

These are applications that will be able to use OAuth2 to get an API key
from Coder.

* Add endpoints for managing OAuth2 applications

These let you add, update, and remove OAuth2 applications.

* Add frontend for managing OAuth2 applications
2023-12-21 21:38:42 +00:00
Kayla Washburn e044d3b752 fix: add additional theme colors (#11313) 2023-12-21 12:59:39 -07:00
Jon Ayers 0b7d68dc3f chore: remove template_update_policies experiment (#11250) 2023-12-21 13:39:33 -06:00
Muhammad Atif Ali 5b071f4d94 feat(examples/templates): add GCP VM devcontainer template (#11246) 2023-12-21 13:01:10 +00:00
Spike Curtis 52b87a28b0 fix: stop printing warnings on external provisioner daemon command (#11309)
fixes #11307
2023-12-21 16:55:34 +04:00
Spike Curtis db9104c02e fix: avoid panic on nil connection (#11305)
Related to https://github.com/coder/coder/actions/runs/7286675441/job/19855871305

Fixes a panic if the listener returns an error, which can obfuscate the underlying problem and cause unrelated tests to be marked failed.
2023-12-21 14:26:11 +04:00
Steven Masley fe867d02e0 fix: correct perms for forbidden error in TemplateScheduleStore.Load (#11286)
* chore: TemplateScheduleStore.Load() throwing forbidden error
* fix: workspace agent scope to include template
2023-12-20 11:38:49 -06:00
Kira Pilot 20dff2aa5d added react query dev tools (#11293) 2023-12-20 10:08:51 -05:00
Ben Potter 19e4a86711 docs: add guidelines for debugging group sync (#11296)
* docs: add guidelines for debugging group sync

* fmt
2023-12-20 12:52:07 +00:00
Bruno Quaresma e2e56d7d4f refactor(site): move workspace schedule controls to its own component (#11281) 2023-12-20 08:46:18 -03:00
Cian Johnston bfc588955c ci: make test-go-pg depend on sqlc-vet (#11288) 2023-12-20 08:47:47 +00:00
Muhammad Atif Ali 3ffe7f55aa feat(examples/templates): add aws vm devcontainer template (#11248)
* feat(examples/templates): add aws vm devcontainer template

* Create README.md

* add code-server

* fix code-server

* `make fmt`

* Add files via upload

* Update README.md

* fix typo and persist workspace

* always land in the repo directory
2023-12-20 08:24:45 +03:00
Kayla Washburn 97f7a35a47 feat: add light theme (#11266) 2023-12-19 17:03:00 -07:00
Bruno Quaresma e0d34ca6f7 fix(site): fix error when loading workspaces with dormant (#11291) 2023-12-19 20:42:07 -03:00
Steven Masley 24080b121c feat: enable csrf token header (#11283)
* feat: enable csrf token header

* Exempt external auth requets
* ensure dev server bypasses CSRF
* external auth is just get requests
* Add some more routes
* Extra assurance nothing breaks
2023-12-19 15:42:05 -06:00
Steven Masley fbda21a9f2 feat: move moons experiment to ga (released) (#11285)
* feat: release moons experiment as ga
2023-12-19 14:40:22 -06:00
Steven Masley e8be092af0 chore: add sqlc push action on releases (#11171)
* add sqlc push action on releases
* Make sqlc push optional
2023-12-19 20:31:55 +00:00
Steven Masley c1451ca4da chore: implement yaml parsing for external auth configs (#11268)
* chore: yaml parsing for external auth configs
* Also unmarshal and check the output again
2023-12-19 18:09:45 +00:00
dependabot[bot] 016b3ef5a2 chore: bump golang.org/x/crypto from 0.15.0 to 0.17.0 (#11274)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.15.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.15.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-19 20:52:43 +03:00
Cian Johnston d2d7628522 fix(enterprise/cli): add CODER_PROVISIONER_DAEMON_LOG_* options (#11279)
- Extracts cli.BuildLogger to clilog package
- Updates existing usage of cli.BuildLogger and removes it
- Use clilog to initialize provisionerd logger
2023-12-19 16:49:50 +00:00
Bruno Quaresma 7c4fbe5bae refactor(site): make HelpTooltip easier to reuse and compose (#11242) 2023-12-19 10:43:23 -03:00
Spike Curtis f2606a78dd fix: avoid converting nil node
fixes: #11276
2023-12-19 13:38:15 +04:00
Stephen Kirby 83e1349c2c moved docker installation warning to install/docker (#11273) 2023-12-18 18:19:20 -06:00
MarkE 280d38d4b8 added UI as Dashboard synonym (#11271) 2023-12-18 17:13:07 -06:00
Kayla Washburn 3ab4800a18 chore: clean up lint (#11270) 2023-12-18 14:59:39 -07:00
Bruno Quaresma e84d89353f fix(site): fix template editor filetree navigation (#11260)
Close https://github.com/coder/coder/issues/11203
2023-12-18 14:21:24 -03:00
Cian Johnston ff61475239 fix(coderd/provisionerdserver): use s.timeNow (#11267) 2023-12-18 17:11:50 +00:00
Steven Masley c35b560c87 chore: fix flake, use time closer to actual test (#11240)
* chore: fix flake, use time closer to actual test

The tests were queued, and the autostart time was being set
to the time the table was created, not when the test was actually
being run. This diff was causing failures in CI
2023-12-18 10:55:46 -06:00
Cian Johnston 213b768785 feat(coderd): insert provisioner daemons (#11207)
* Adds UpdateProvisionerDaemonLastSeenAt
* Adds heartbeat to provisioner daemons
* Inserts provisioner daemons to database upon start
* Ensures TagOwner is an empty string and not nil
* Adds COALESCE() in idx_provisioner_daemons_name_owner_key
2023-12-18 16:44:52 +00:00
Steven Masley a6901ae2c5 chore: fix race in cron close behavior (TestAgent_WriteVSCodeConfigs) (#11243)
* chore: add unit test to excercise flake
* Implement a *fix for cron stop() before run()

This fix still has a race condition. I do not see a clean solution
without modifying the cron libary. The cron library uses a boolean
to indicate running, and that boolean needs to be set to "true"
before we call "Close()". Or "Close()" should prevent "Run()"
from doing anything.

In either case, this solves the issue for a niche unit test bug
in which the test finishes, calling Close(), before there was
an oppertunity to start the go routine. It probably isn't worth
a lot of time investment, and this fix will suffice
2023-12-18 09:26:40 -06:00
Jon Ayers 56cbd47082 chore: fix TestWorkspaceAutobuild/DormancyThresholdOK flake (#11251) 2023-12-18 09:23:06 -06:00
Muhammad Atif Ali 45e9d93d37 chore: remove unused input from deploy-pr workflow (#11259) 2023-12-18 17:32:53 +03:00
Muhammad Atif Ali 5647e87207 ci: drop chocolatey from ci (#11245) 2023-12-18 17:31:35 +03:00
Dean Sheather 307186325f fix: avoid db import in slim builds (#11258) 2023-12-19 00:09:22 +10:00
dependabot[bot] 28a0242c27 ci: bump the github-actions group with 4 updates (#11256)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-18 13:30:18 +00:00
Dean Sheather e46431078c feat: add AgentAPI using DRPC (#10811)
Co-authored-by: Spike Curtis <spike@coder.com>
2023-12-18 22:53:28 +10:00
Cian Johnston eb781751b8 ci: update flux to 2.2.1 (#11253) 2023-12-18 09:29:46 +00:00
Muhammad Atif Ali 838ab8de7e docs: fix a broken link (#11254) 2023-12-18 09:28:55 +00:00
Ben Potter 2e86b76fb8 docs: improve structure for example templates (#9842)
Co-authored-by: Kyle Carberry <kyle@carberry.com>
Co-authored-by: Muhammad Atif Ali <atif@coder.com>
Co-authored-by: Muhammad Atif Ali <me@matifali.dev>
2023-12-17 17:05:13 +03:00
Steven Masley 3f6096b0d7 chore: unit test to enforce authorized queries match args (#11211)
* chore: unit test to enforce authorized queries match args
* Also check querycontext arguments
2023-12-15 20:31:07 +00:00
Garrett Delfosse 7924bb2a56 feat!: move workspace renames behind flag, disable by default (#11189) 2023-12-15 13:38:47 -05:00
Steven Masley e63de9a259 chore: enforcement of dbauthz tests was broken (#11218)
* chore: enforcement of dbauthz tests was broken

Implemented missing tests to catch back up

---------

Co-authored-by: Cian Johnston <cian@coder.com>
2023-12-15 18:30:21 +00:00
Stephen Kirby 0801760956 docs: add guides section (#11199)
* setup manifest

* added okta guide from steven M

* improved index by adding children

* changed icon to notes.svg

* added meta guide, fixed profile photo fmt
2023-12-15 11:10:41 -06:00
Ravindra Shinde a495952349 Upgrade code-server version to 4.19.1 (#11233) 2023-12-15 14:21:07 +00:00
Marcin Tojek 58c2ce17da refactor(cli): load template variables (#11234) 2023-12-15 14:55:24 +01:00
Cian Johnston fa91992976 ci: add audit docs gen dependency on db gen (#11231)
Audit docs gen depends on queries.sql.go so adding an explicit dependency
2023-12-15 11:49:19 +00:00
Marcin Tojek 89d8a293f0 fix: tar: do not archive .tfvars (#11208) 2023-12-15 11:15:12 +01:00
Spike Curtis 211e59bf65 feat: add tailnet v2 API support to coordinate endpoint (#11228)
closes #10532

Adds v2 support to the /coordinate endpoint via a query parameter.

v1 already has test cases, and we haven't implemented v2 at the client yet, so the only new test case is an unsupported version.
2023-12-15 14:10:24 +04:00
Cian Johnston a41cbb0f03 chore(dogfood): align Terraform version to that of dockerfile.base (#11227) 2023-12-15 10:02:59 +00:00
Dean Sheather 1e49190e12 feat: add server flag to disable user custom quiet hours (#11124) 2023-12-15 19:33:51 +10:00
Spike Curtis a58e4febb9 feat: add tailnet v2 Service and Client (#11225)
Part of #10532

Adds a tailnet ClientService that accepts a net.Conn and serves v1 or v2 of the tailnet API.

Also adds a DRPCService that implements the DRPC interface for the v2 API.  This component is within the ClientService, but needs to be reusable and exported so that we can also embed it in the Agent API.

Finally, includes a NewDRPCClient function that takes a net.Conn and runs dRPC in yamux over it on the client side.
2023-12-15 12:48:39 +04:00
Spike Curtis 9a4e1100fa chore: move drpc transport tools to codersdk/drpc (#11224)
Part of #10532

DRPC transport over yamux and in-mem pipes was previously only used on the provisioner APIs, but now will also be used in tailnet.  Moved to subpackage of codersdk to avoid import loops.
2023-12-15 12:41:39 +04:00
Dean Sheather b36071c6bb feat: allow templates to specify max_ttl or autostop_requirement (#10920) 2023-12-15 18:27:56 +10:00
Spike Curtis 30f032d282 feat: add tailnet ValidateVersion (#11223)
Part of #10532

Adds a method to validate a requested version of the tailnet API
2023-12-15 11:49:30 +04:00
Spike Curtis ad3fed72bc chore: rename Coordinator to CoordinatorV1 (#11222)
Renames the tailnet.Coordinator to represent both v1 and v2 APIs, so that we can use this interface for the main atomic pointer.

Part of #10532
2023-12-15 11:38:12 +04:00
Spike Curtis 545cb9a7cc fix: wait for coordinator in Test_agentIsLegacy (#11214)
Fixes flake https://github.com/coder/coder/runs/19639217635

AGPL coordinator used to process node updates for single_tailnet synchronously, but it's been refactored to process async, so in this test we need to wait for it to be processed.
2023-12-15 07:21:18 +04:00
2075 changed files with 120942 additions and 50336 deletions
+6
View File
@@ -0,0 +1,6 @@
# Ignore all files and folders
**
# Include flake.nix and flake.lock
!flake.nix
!flake.lock
+3
View File
@@ -6,9 +6,12 @@ coderd/apidoc/swagger.json linguist-generated=true
coderd/database/dump.sql linguist-generated=true
peerbroker/proto/*.go linguist-generated=true
provisionerd/proto/*.go linguist-generated=true
provisionerd/proto/version.go linguist-generated=false
provisionersdk/proto/*.go linguist-generated=true
*.tfplan.json linguist-generated=true
*.tfstate.json linguist-generated=true
*.tfstate.dot linguist-generated=true
*.tfplan.dot linguist-generated=true
site/e2e/provisionerGenerated.ts linguist-generated=true
site/src/api/typesGenerated.ts linguist-generated=true
site/src/pages/SetupPage/countries.tsx linguist-generated=true
+2 -2
View File
@@ -4,12 +4,12 @@ description: |
inputs:
version:
description: "The Go version to use."
default: "1.21.5"
default: "1.21.9"
runs:
using: "composite"
steps:
- name: Setup Go
uses: buildjet/setup-go@v4
uses: buildjet/setup-go@v5
with:
go-version: ${{ inputs.version }}
+3 -3
View File
@@ -11,13 +11,13 @@ runs:
using: "composite"
steps:
- name: Install pnpm
uses: pnpm/action-setup@v2
uses: pnpm/action-setup@v3
with:
version: 8
- name: Setup Node
uses: buildjet/setup-node@v3
uses: buildjet/setup-node@v4.0.1
with:
node-version: 18.17.0
node-version: 18.19.0
# See https://github.com/actions/setup-node#caching-global-packages-data
cache: "pnpm"
cache-dependency-path: ${{ inputs.directory }}/pnpm-lock.yaml
+1 -1
View File
@@ -7,4 +7,4 @@ runs:
- name: Setup sqlc
uses: sqlc-dev/setup-sqlc@v4
with:
sqlc-version: "1.24.0"
sqlc-version: "1.25.0"
+1 -1
View File
@@ -7,5 +7,5 @@ runs:
- name: Install Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.5.7
terraform_version: 1.7.5
terraform_wrapper: false
-43
View File
@@ -1,43 +0,0 @@
codecov:
require_ci_to_pass: false
notify:
after_n_builds: 5
comment: false
github_checks:
annotations: false
coverage:
range: 50..75
round: down
precision: 2
status:
patch:
default:
informational: yes
project:
default:
target: 65%
informational: true
ignore:
# This is generated code.
- coderd/database/models.go
- coderd/database/queries.sql.go
- coderd/database/databasefake
# These are generated or don't require tests.
- cmd
- coderd/tunnel
- coderd/database/dump
- coderd/database/postgres
- peerbroker/proto
- provisionerd/proto
- provisionersdk/proto
- scripts
- site/.storybook
- rules.go
# Packages used for writing tests.
- cli/clitest
- coderd/coderdtest
- pty/ptytest
+1 -4
View File
@@ -38,15 +38,12 @@ updates:
commit-message:
prefix: "chore"
labels: []
open-pull-requests-limit: 15
ignore:
# Ignore patch updates for all dependencies
- dependency-name: "*"
update-types:
- version-update:semver-patch
groups:
go:
patterns:
- "*"
# Update our Dockerfile.
- package-ecosystem: "docker"
+34
View File
@@ -0,0 +1,34 @@
app = "jnb-coder"
primary_region = "jnb"
[experimental]
entrypoint = ["/bin/sh", "-c", "CODER_DERP_SERVER_RELAY_URL=\"http://[${FLY_PRIVATE_IP}]:3000\" /opt/coder wsproxy server"]
auto_rollback = true
[build]
image = "ghcr.io/coder/coder-preview:main"
[env]
CODER_ACCESS_URL = "https://jnb.fly.dev.coder.com"
CODER_HTTP_ADDRESS = "0.0.0.0:3000"
CODER_PRIMARY_ACCESS_URL = "https://dev.coder.com"
CODER_WILDCARD_ACCESS_URL = "*--apps.jnb.fly.dev.coder.com"
CODER_VERBOSE = "true"
[http_service]
internal_port = 3000
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
# Ref: https://fly.io/docs/reference/configuration/#http_service-concurrency
[http_service.concurrency]
type = "requests"
soft_limit = 50
hard_limit = 100
[[vm]]
cpu_kind = "shared"
cpus = 2
memory_mb = 512
+7
View File
@@ -13,6 +13,7 @@ primary_region = "cdg"
CODER_HTTP_ADDRESS = "0.0.0.0:3000"
CODER_PRIMARY_ACCESS_URL = "https://dev.coder.com"
CODER_WILDCARD_ACCESS_URL = "*--apps.paris.fly.dev.coder.com"
CODER_VERBOSE = "true"
[http_service]
internal_port = 3000
@@ -21,6 +22,12 @@ primary_region = "cdg"
auto_start_machines = true
min_machines_running = 0
# Ref: https://fly.io/docs/reference/configuration/#http_service-concurrency
[http_service.concurrency]
type = "requests"
soft_limit = 50
hard_limit = 100
[[vm]]
cpu_kind = "shared"
cpus = 2
@@ -13,6 +13,7 @@ primary_region = "gru"
CODER_HTTP_ADDRESS = "0.0.0.0:3000"
CODER_PRIMARY_ACCESS_URL = "https://dev.coder.com"
CODER_WILDCARD_ACCESS_URL = "*--apps.sao-paulo.fly.dev.coder.com"
CODER_VERBOSE = "true"
[http_service]
internal_port = 3000
@@ -21,6 +22,12 @@ primary_region = "gru"
auto_start_machines = true
min_machines_running = 0
# Ref: https://fly.io/docs/reference/configuration/#http_service-concurrency
[http_service.concurrency]
type = "requests"
soft_limit = 50
hard_limit = 100
[[vm]]
cpu_kind = "shared"
cpus = 2
+7
View File
@@ -13,6 +13,7 @@ primary_region = "syd"
CODER_HTTP_ADDRESS = "0.0.0.0:3000"
CODER_PRIMARY_ACCESS_URL = "https://dev.coder.com"
CODER_WILDCARD_ACCESS_URL = "*--apps.sydney.fly.dev.coder.com"
CODER_VERBOSE = "true"
[http_service]
internal_port = 3000
@@ -21,6 +22,12 @@ primary_region = "syd"
auto_start_machines = true
min_machines_running = 0
# Ref: https://fly.io/docs/reference/configuration/#http_service-concurrency
[http_service.concurrency]
type = "requests"
soft_limit = 50
hard_limit = 100
[[vm]]
cpu_kind = "shared"
cpus = 2
+3 -4
View File
@@ -88,10 +88,9 @@ provider "kubernetes" {
data "coder_workspace" "me" {}
resource "coder_agent" "main" {
os = "linux"
arch = "amd64"
startup_script_timeout = 180
startup_script = <<-EOT
os = "linux"
arch = "amd64"
startup_script = <<-EOT
set -e
# install and start code-server
+155 -174
View File
@@ -46,7 +46,7 @@ jobs:
fetch-depth: 1
# For pull requests it's not necessary to checkout the code
- name: check changed files
uses: dorny/paths-filter@v2
uses: dorny/paths-filter@v3
id: filter
with:
filters: |
@@ -60,10 +60,7 @@ jobs:
- "examples/lima/**"
db:
- "**.sql"
- "coderd/database/queries/**"
- "coderd/database/migrations"
- "coderd/database/sqlc.yaml"
- "coderd/database/dump.sql"
- "coderd/database/**"
go:
- "**.sql"
- "**.go"
@@ -129,12 +126,13 @@ jobs:
- name: Get golangci-lint cache dir
run: |
go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.53.2
linter_ver=$(egrep -o 'GOLANGCI_LINT_VERSION=\S+' dogfood/Dockerfile | cut -d '=' -f 2)
go install github.com/golangci/golangci-lint/cmd/golangci-lint@v$linter_ver
dir=$(golangci-lint cache status | awk '/Dir/ { print $2 }')
echo "LINT_CACHE_DIR=$dir" >> $GITHUB_ENV
- name: golangci-lint cache
uses: buildjet/cache@v3
uses: buildjet/cache@v4
with:
path: |
${{ env.LINT_CACHE_DIR }}
@@ -144,7 +142,7 @@ jobs:
# Check for any typos
- name: Check for typos
uses: crate-ci/typos@v1.16.24
uses: crate-ci/typos@v1.20.10
with:
config: .github/workflows/typos.toml
@@ -157,7 +155,7 @@ jobs:
# Needed for helm chart linting
- name: Install helm
uses: azure/setup-helm@v3
uses: azure/setup-helm@v4
with:
version: v3.9.2
@@ -185,13 +183,16 @@ jobs:
- name: Setup sqlc
uses: ./.github/actions/setup-sqlc
- name: Setup Terraform
uses: ./.github/actions/setup-tf
- name: go install tools
run: |
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.30
go install storj.io/drpc/cmd/protoc-gen-go-drpc@v0.0.33
go install golang.org/x/tools/cmd/goimports@latest
go install github.com/mikefarah/yq/v4@v4.30.6
go install github.com/golang/mock/mockgen@v1.6.0
go install go.uber.org/mock/mockgen@v0.4.0
- name: Install Protoc
run: |
@@ -204,7 +205,9 @@ jobs:
popd
- name: make gen
run: "make --output-sync -j -B gen"
# no `-j` flag as `make` fails with:
# coderd/rbac/object_gen.go:1:1: syntax error: package statement must be first
run: "make --output-sync -B gen"
- name: Check for unstaged files
run: ./scripts/check_unstaged.sh
@@ -224,11 +227,11 @@ jobs:
uses: ./.github/actions/setup-node
- name: Setup Go
uses: buildjet/setup-go@v4
uses: buildjet/setup-go@v5
with:
# This doesn't need caching. It's super fast anyways!
cache: false
go-version: 1.21.5
go-version: 1.21.9
- name: Install shfmt
run: go install mvdan.cc/sh/v3/cmd/shfmt@v3.7.0
@@ -269,16 +272,6 @@ jobs:
id: test
shell: bash
run: |
# Code coverage is more computationally expensive and also
# prevents test caching, so we disable it on alternate operating
# systems.
if [ "${{ matrix.os }}" == "ubuntu-latest" ]; then
echo "cover=true" >> $GITHUB_OUTPUT
export COVERAGE_FLAGS='-covermode=atomic -coverprofile="gotests.coverage" -coverpkg=./...'
else
echo "cover=false" >> $GITHUB_OUTPUT
fi
# if macOS, install google-chrome for scaletests. As another concern,
# should we really have this kind of external dependency requirement
# on standard CI?
@@ -297,7 +290,7 @@ jobs:
fi
export TS_DEBUG_DISCO=true
gotestsum --junitfile="gotests.xml" --jsonfile="gotests.json" \
--packages="./..." -- $PARALLEL_FLAG -short -failfast $COVERAGE_FLAGS
--packages="./..." -- $PARALLEL_FLAG -short -failfast
- name: Upload test stats to Datadog
timeout-minutes: 1
@@ -307,22 +300,10 @@ jobs:
with:
api-key: ${{ secrets.DATADOG_API_KEY }}
- name: Check code coverage
uses: codecov/codecov-action@v3
# This action has a tendency to error out unexpectedly, it has
# the `fail_ci_if_error` option that defaults to `false`, but
# that is no guarantee, see:
# https://github.com/codecov/codecov-action/issues/788
continue-on-error: true
if: steps.test.outputs.cover && github.actor != 'dependabot[bot]' && !github.event.pull_request.head.repo.fork
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./gotests.coverage
flags: unittest-go-${{ matrix.os }}
test-go-pg:
runs-on: ${{ github.repository_owner == 'coder' && 'buildjet-8vcpu-ubuntu-2204' || 'ubuntu-latest' }}
needs: changes
needs:
- changes
if: needs.changes.outputs.go == 'true' || needs.changes.outputs.ci == 'true' || github.ref == 'refs/heads/main'
# This timeout must be greater than the timeout set by `go test` in
# `make test-postgres` to ensure we receive a trace of running
@@ -354,19 +335,6 @@ jobs:
with:
api-key: ${{ secrets.DATADOG_API_KEY }}
- name: Check code coverage
uses: codecov/codecov-action@v3
# This action has a tendency to error out unexpectedly, it has
# the `fail_ci_if_error` option that defaults to `false`, but
# that is no guarantee, see:
# https://github.com/codecov/codecov-action/issues/788
continue-on-error: true
if: github.actor != 'dependabot[bot]' && !github.event.pull_request.head.repo.fork
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./gotests.coverage
flags: unittest-go-postgres-linux
test-go-race:
runs-on: ${{ github.repository_owner == 'coder' && 'buildjet-8vcpu-ubuntu-2204' || 'ubuntu-latest' }}
needs: changes
@@ -413,24 +381,20 @@ jobs:
- run: pnpm test:ci --max-workers $(nproc)
working-directory: site
- name: Check code coverage
uses: codecov/codecov-action@v3
# This action has a tendency to error out unexpectedly, it has
# the `fail_ci_if_error` option that defaults to `false`, but
# that is no guarantee, see:
# https://github.com/codecov/codecov-action/issues/788
continue-on-error: true
if: github.actor != 'dependabot[bot]' && !github.event.pull_request.head.repo.fork
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./site/coverage/lcov.info
flags: unittest-js
test-e2e:
runs-on: ${{ github.repository_owner == 'coder' && 'buildjet-16vcpu-ubuntu-2204' || 'ubuntu-latest' }}
needs: changes
if: needs.changes.outputs.go == 'true' || needs.changes.outputs.ts == 'true' || needs.changes.outputs.ci == 'true' || github.ref == 'refs/heads/main'
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
variant:
- enterprise: false
name: test-e2e
- enterprise: true
name: test-e2e-enterprise
name: ${{ matrix.variant.name }}
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -443,52 +407,48 @@ jobs:
- name: Setup Go
uses: ./.github/actions/setup-go
- name: Setup Terraform
uses: ./.github/actions/setup-tf
# Assume that the checked-in versions are up-to-date
- run: make gen/mark-fresh
name: make gen
- name: go install tools
run: |
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.30
go install storj.io/drpc/cmd/protoc-gen-go-drpc@v0.0.33
go install golang.org/x/tools/cmd/goimports@latest
go install github.com/mikefarah/yq/v4@v4.30.6
go install github.com/golang/mock/mockgen@v1.6.0
- name: Install Protoc
run: |
mkdir -p /tmp/proto
pushd /tmp/proto
curl -L -o protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v23.3/protoc-23.3-linux-x86_64.zip
unzip protoc.zip
cp -r ./bin/* /usr/local/bin
cp -r ./include /usr/local/bin/include
popd
- name: Build
run: |
make -B site/out/index.html
- run: pnpm build
working-directory: site
- run: pnpm playwright:install
working-directory: site
- run: pnpm playwright:test --workers 1
# Run tests that don't require an enterprise license without an enterprise license
- run: pnpm playwright:test --forbid-only --workers 1
if: ${{ !matrix.variant.enterprise }}
env:
DEBUG: pw:api
working-directory: site
# Run all of the tests with an enterprise license
- run: pnpm playwright:test --forbid-only --workers 1
if: ${{ matrix.variant.enterprise }}
env:
DEBUG: pw:api
CODER_E2E_ENTERPRISE_LICENSE: ${{ secrets.CODER_E2E_ENTERPRISE_LICENSE }}
CODER_E2E_REQUIRE_ENTERPRISE_TESTS: "1"
working-directory: site
# Temporarily allow these to fail so that I can gather data about which
# tests are failing.
continue-on-error: true
- name: Upload Playwright Failed Tests
if: always() && github.actor != 'dependabot[bot]' && runner.os == 'Linux' && !github.event.pull_request.head.repo.fork
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: failed-test-videos
name: failed-test-videos${{ matrix.variant.enterprise && '-enterprise' || '-agpl' }}
path: ./site/test-results/**/*.webm
retention-days: 7
- name: Upload pprof dumps
if: always() && github.actor != 'dependabot[bot]' && runner.os == 'Linux' && !github.event.pull_request.head.repo.fork
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: debug-pprof-dumps
name: debug-pprof-dumps${{ matrix.variant.enterprise && '-enterprise' || '-agpl' }}
path: ./site/test-results/**/debug-pprof-*.txt
retention-days: 7
@@ -518,7 +478,8 @@ jobs:
NODE_OPTIONS: "--max_old_space_size=4096"
STORYBOOK: true
with:
buildScriptName: "storybook:build"
# Do a fast, testing build for change previews
buildScriptName: "storybook:ci"
exitOnceUploaded: true
# This will prevent CI from failing when Chromatic detects visual changes
exitZeroOnChanges: true
@@ -532,6 +493,8 @@ jobs:
# Run TurboSnap to trace file dependencies to related stories
# and tell chromatic to only take snapshots of relevent stories
onlyChanged: true
# Avoid uploading single files, because that's very slow
zip: true
# This is a separate step for mainline only that auto accepts and changes
# instead of holding CI up. Since we squash/merge, this is defensive to
@@ -549,6 +512,7 @@ jobs:
autoAcceptChanges: true
# This will prevent CI from failing when Chromatic detects visual changes
exitZeroOnChanges: true
# Do a full build with documentation for mainline builds
buildScriptName: "storybook:build"
projectToken: 695c25b6cb65
workingDir: "./site"
@@ -556,6 +520,8 @@ jobs:
# Run TurboSnap to trace file dependencies to related stories
# and tell chromatic to only take snapshots of relevent stories
onlyChanged: true
# Avoid uploading single files, because that's very slow
zip: true
offlinedocs:
name: offlinedocs
@@ -594,7 +560,7 @@ jobs:
go install storj.io/drpc/cmd/protoc-gen-go-drpc@v0.0.33
go install golang.org/x/tools/cmd/goimports@latest
go install github.com/mikefarah/yq/v4@v4.30.6
go install github.com/golang/mock/mockgen@v1.6.0
go install go.uber.org/mock/mockgen@v0.4.0
- name: Setup sqlc
uses: ./.github/actions/setup-sqlc
@@ -610,8 +576,10 @@ jobs:
pnpm lint
- name: Build
# no `-j` flag as `make` fails with:
# coderd/rbac/object_gen.go:1:1: syntax error: package statement must be first
run: |
make -j build/coder_docs_"$(./scripts/version.sh)".tgz
make build/coder_docs_"$(./scripts/version.sh)".tgz
required:
runs-on: ubuntu-latest
@@ -626,6 +594,7 @@ jobs:
- test-e2e
- offlinedocs
- sqlc-vet
- dependency-license-review
# Allow this job to run even if the needed jobs fail, are skipped or
# cancelled.
if: always()
@@ -642,6 +611,7 @@ jobs:
echo "- test-js: ${{ needs.test-js.result }}"
echo "- test-e2e: ${{ needs.test-e2e.result }}"
echo "- offlinedocs: ${{ needs.offlinedocs.result }}"
echo "- dependency-license-review: ${{ needs.dependency-license-review.result }}"
echo
# We allow skipped jobs to pass, but not failed or cancelled jobs.
@@ -657,7 +627,7 @@ jobs:
# to main branch. We are only building this for amd64 platform. (>95% pulls
# are for amd64)
needs: changes
if: github.ref == 'refs/heads/main' && needs.changes.outputs.docs-only == 'false'
if: github.ref == 'refs/heads/main' && needs.changes.outputs.docs-only == 'false' && !github.event.pull_request.head.repo.fork
runs-on: ${{ github.repository_owner == 'coder' && 'buildjet-8vcpu-ubuntu-2204' || 'ubuntu-latest' }}
env:
DOCKER_CLI_EXPERIMENTAL: "enabled"
@@ -683,7 +653,7 @@ jobs:
uses: ./.github/actions/setup-go
- name: Install nfpm
run: go install github.com/goreleaser/nfpm/v2/cmd/nfpm@v2.16.0
run: go install github.com/goreleaser/nfpm/v2/cmd/nfpm@v2.35.1
- name: Install zstd
run: sudo apt-get install -y zstd
@@ -694,47 +664,71 @@ jobs:
go mod download
version="$(./scripts/version.sh)"
tag="main-$(echo "$version" | sed 's/+/-/g')"
echo "tag=$tag" >> $GITHUB_OUTPUT
make gen/mark-fresh
make -j \
build/coder_linux_amd64 \
build/coder_linux_{amd64,arm64,armv7} \
build/coder_"$version"_windows_amd64.zip \
build/coder_"$version"_linux_amd64.{tar.gz,deb}
- name: Build and Push Linux amd64 Docker Image
- name: Build Linux Docker images
id: build-docker
env:
CODER_IMAGE_BASE: ghcr.io/coder/coder-preview
CODER_IMAGE_TAG_PREFIX: main
DOCKER_CLI_EXPERIMENTAL: "enabled"
run: |
set -euxo pipefail
# build Docker images for each architecture
version="$(./scripts/version.sh)"
tag="main-$(echo "$version" | sed 's/+/-/g')"
export CODER_IMAGE_BUILD_BASE_TAG="$(CODER_IMAGE_BASE=coder-base ./scripts/image_tag.sh --version "$version")"
./scripts/build_docker.sh \
--arch amd64 \
--target "ghcr.io/coder/coder-preview:$tag" \
--version $version \
--push \
build/coder_linux_amd64
# Tag as main
docker tag "ghcr.io/coder/coder-preview:$tag" ghcr.io/coder/coder-preview:main
docker push ghcr.io/coder/coder-preview:main
# Store the tag in an output variable so we can use it in other jobs
echo "tag=$tag" >> $GITHUB_OUTPUT
# build images for each architecture
make -j build/coder_"$version"_linux_{amd64,arm64,armv7}.tag
# only push if we are on main branch
if [ "${{ github.ref }}" == "refs/heads/main" ]; then
# build and push multi-arch manifest, this depends on the other images
# being pushed so will automatically push them
make -j push/build/coder_"$version"_linux_{amd64,arm64,armv7}.tag
# Define specific tags
tags=("$tag" "main" "latest")
# Create and push a multi-arch manifest for each tag
# we are adding `latest` tag and keeping `main` for backward
# compatibality
for t in "${tags[@]}"; do
./scripts/build_docker_multiarch.sh \
--push \
--target "ghcr.io/coder/coder-preview:$t" \
--version $version \
$(cat build/coder_"$version"_linux_{amd64,arm64,armv7}.tag)
done
fi
- name: Prune old images
uses: vlaurin/action-ghcr-prune@v0.5.0
if: github.ref == 'refs/heads/main'
uses: vlaurin/action-ghcr-prune@v0.6.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
organization: coder
container: coder-preview
keep-younger-than: 7 # days
keep-tags: latest
keep-tags-regexes: ^pr
prune-tags-regexes: ^main-
prune-tags-regexes: |
^main-
^v
prune-untagged: true
- name: Upload build artifacts
uses: actions/upload-artifact@v3
if: github.ref == 'refs/heads/main'
uses: actions/upload-artifact@v4
with:
name: coder
path: |
@@ -775,7 +769,7 @@ jobs:
uses: fluxcd/flux2/action@main
with:
# Keep this up to date with the version of flux installed in dogfood cluster
version: "2.2.0"
version: "2.2.1"
- name: Get Cluster Credentials
uses: "google-github-actions/get-gke-credentials@v2"
@@ -827,68 +821,14 @@ jobs:
flyctl deploy --image "$IMAGE" --app paris-coder --config ./.github/fly-wsproxies/paris-coder.toml --env "CODER_PROXY_SESSION_TOKEN=$TOKEN_PARIS" --yes
flyctl deploy --image "$IMAGE" --app sydney-coder --config ./.github/fly-wsproxies/sydney-coder.toml --env "CODER_PROXY_SESSION_TOKEN=$TOKEN_SYDNEY" --yes
flyctl deploy --image "$IMAGE" --app sao-paulo-coder --config ./.github/fly-wsproxies/sao-paulo-coder.toml --env "CODER_PROXY_SESSION_TOKEN=$TOKEN_SAO_PAULO" --yes
flyctl deploy --image "$IMAGE" --app jnb-coder --config ./.github/fly-wsproxies/jnb-coder.toml --env "CODER_PROXY_SESSION_TOKEN=$TOKEN_JNB" --yes
env:
FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
IMAGE: ${{ needs.build.outputs.IMAGE }}
TOKEN_PARIS: ${{ secrets.FLY_PARIS_CODER_PROXY_SESSION_TOKEN }}
TOKEN_SYDNEY: ${{ secrets.FLY_SYDNEY_CODER_PROXY_SESSION_TOKEN }}
TOKEN_SAO_PAULO: ${{ secrets.FLY_SAO_PAULO_CODER_PROXY_SESSION_TOKEN }}
deploy-legacy-proxies:
runs-on: ubuntu-latest
timeout-minutes: 30
needs: build
if: github.ref == 'refs/heads/main' && !github.event.pull_request.head.repo.fork
permissions:
contents: read
id-token: write
steps:
- name: Authenticate to Google Cloud
uses: google-github-actions/auth@v2
with:
workload_identity_provider: projects/573722524737/locations/global/workloadIdentityPools/github/providers/github
service_account: coder-ci@coder-dogfood.iam.gserviceaccount.com
- name: Set up Google Cloud SDK
uses: google-github-actions/setup-gcloud@v2
- name: Download build artifacts
uses: actions/download-artifact@v3
with:
name: coder
path: ./build
- name: Install Release
run: |
set -euo pipefail
regions=(
# gcp-region-id instance-name systemd-service-name
"australia-southeast1-b coder-sydney coder-workspace-proxy"
"europe-west3-c coder-europe coder-workspace-proxy"
"southamerica-east1-b coder-brazil coder-workspace-proxy"
)
deb_pkg=$(find ./build -name "coder_*_linux_amd64.deb" -print -quit)
if [ -z "$deb_pkg" ]; then
echo "deb package $deb_pkg not found"
ls -l ./build
exit 1
fi
gcloud config set project coder-dogfood
for region in "${regions[@]}"; do
echo "::group::$region"
set -- $region
set -x
gcloud config set compute/zone "$1"
gcloud compute scp "$deb_pkg" "${2}:/tmp/coder.deb"
gcloud compute ssh "$2" -- /bin/sh -c "set -eux; sudo dpkg -i --force-confdef /tmp/coder.deb; sudo systemctl daemon-reload; sudo service '$3' restart"
set +x
echo "::endgroup::"
done
TOKEN_JNB: ${{ secrets.FLY_JNB_CODER_PROXY_SESSION_TOKEN }}
# sqlc-vet runs a postgres docker container, runs Coder migrations, and then
# runs sqlc-vet to ensure all queries are valid. This catches any mistakes
@@ -896,7 +836,7 @@ jobs:
sqlc-vet:
runs-on: ${{ github.repository_owner == 'coder' && 'buildjet-8vcpu-ubuntu-2204' || 'ubuntu-latest' }}
needs: changes
if: needs.changes.outputs.db == 'true' || github.ref == 'refs/heads/main'
if: needs.changes.outputs.db == 'true' || needs.changes.outputs.ci == 'true' || github.ref == 'refs/heads/main'
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -912,3 +852,44 @@ jobs:
- name: Setup and run sqlc vet
run: |
make sqlc-vet
# dependency-license-review checks that no license-incompatible dependencies have been introduced.
# This action is not intended to do a vulnerability check since that is handled by a separate action.
dependency-license-review:
runs-on: ubuntu-latest
if: github.ref != 'refs/heads/main'
steps:
- name: "Checkout Repository"
uses: actions/checkout@v4
- name: "Dependency Review"
id: review
# TODO: Replace this with the latest release once https://github.com/actions/dependency-review-action/pull/761 is merged.
uses: actions/dependency-review-action@49fbbe0acb033b7824f26d00b005d7d598d76301
with:
allow-licenses: Apache-2.0, BSD-2-Clause, BSD-3-Clause, CC0-1.0, ISC, MIT, MIT-0, MPL-2.0
allow-dependencies-licenses: "pkg:golang/github.com/pelletier/go-toml/v2"
license-check: true
vulnerability-check: false
- name: "Report"
# make sure this step runs even if the previous failed
if: always()
shell: bash
env:
VULNERABLE_CHANGES: ${{ steps.review.outputs.invalid-license-changes }}
run: |
fields=( "unlicensed" "unresolved" "forbidden" )
# This is unfortunate that we have to do this but the action does not support failing on
# an unknown license. The unknown dependency could easily have a GPL license which
# would be problematic for us.
# Track https://github.com/actions/dependency-review-action/issues/672 for when
# we can remove this brittle workaround.
for field in "${fields[@]}"; do
# Use jq to check if the array is not empty
if [[ $(echo "$VULNERABLE_CHANGES" | jq ".${field} | length") -ne 0 ]]; then
echo "Invalid or unknown licenses detected, contact @sreya to ensure your added dependency falls under one of our allowed licenses."
echo "$VULNERABLE_CHANGES" | jq
exit 1
fi
done
echo "No incompatible licenses detected"
+2 -2
View File
@@ -26,7 +26,7 @@ jobs:
pull-requests: write
steps:
- name: auto-approve dependabot
uses: hmarr/auto-approve-action@v3
uses: hmarr/auto-approve-action@v4
if: github.actor == 'dependabot[bot]'
cla:
@@ -34,7 +34,7 @@ jobs:
steps:
- name: cla
if: (github.event.comment.body == 'recheck' || github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA') || github.event_name == 'pull_request_target'
uses: contributor-assistant/github-action@v2.3.1
uses: contributor-assistant/github-action@v2.3.2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# the below token should have repo scope and must be manually added by you in the repository's secret
+48 -13
View File
@@ -7,16 +7,19 @@ on:
paths:
- "dogfood/**"
- ".github/workflows/dogfood.yaml"
# Uncomment these lines when testing with CI.
# pull_request:
# paths:
# - "dogfood/**"
# - ".github/workflows/dogfood.yaml"
- "flake.lock"
- "flake.nix"
pull_request:
paths:
- "dogfood/**"
- ".github/workflows/dogfood.yaml"
- "flake.lock"
- "flake.nix"
workflow_dispatch:
jobs:
deploy_image:
runs-on: buildjet-4vcpu-ubuntu-2204
build_image:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -33,46 +36,78 @@ jobs:
tag=${tag//\//--}
echo "tag=${tag}" >> $GITHUB_OUTPUT
- name: Set up Depot CLI
uses: depot/setup-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
if: github.ref == 'refs/heads/main'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
- name: Build and push
uses: docker/build-push-action@v5
- name: Build and push Non-Nix image
uses: depot/build-push-action@v1
with:
project: b4q6ltmpzh
token: ${{ secrets.DEPOT_TOKEN }}
buildx-fallback: true
context: "{{defaultContext}}:dogfood"
pull: true
push: true
save: true
push: ${{ github.ref == 'refs/heads/main' }}
tags: "codercom/oss-dogfood:${{ steps.docker-tag-name.outputs.tag }},codercom/oss-dogfood:latest"
cache-from: type=registry,ref=codercom/oss-dogfood:latest
cache-to: type=inline
- name: Build and push Nix image
uses: depot/build-push-action@v1
with:
project: b4q6ltmpzh
token: ${{ secrets.DEPOT_TOKEN }}
buildx-fallback: true
context: "."
file: "dogfood/Dockerfile.nix"
pull: true
save: true
push: ${{ github.ref == 'refs/heads/main' }}
tags: "codercom/oss-dogfood-nix:${{ steps.docker-tag-name.outputs.tag }},codercom/oss-dogfood-nix:latest"
deploy_template:
needs: deploy_image
needs: build_image
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Terraform
uses: ./.github/actions/setup-tf
- name: Terraform init and validate
run: |
cd dogfood
terraform init -upgrade
terraform validate
- name: Get short commit SHA
if: github.ref == 'refs/heads/main'
id: vars
run: echo "sha_short=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
- name: Get latest commit title
if: github.ref == 'refs/heads/main'
id: message
run: echo "pr_title=$(git log --format=%s -n 1 ${{ github.sha }})" >> $GITHUB_OUTPUT
- name: "Get latest Coder binary from the server"
if: github.ref == 'refs/heads/main'
run: |
curl -fsSL "https://dev.coder.com/bin/coder-linux-amd64" -o "./coder"
chmod +x "./coder"
- name: "Push template"
if: github.ref == 'refs/heads/main'
run: |
./coder templates push $CODER_TEMPLATE_NAME --directory $CODER_TEMPLATE_DIR --yes --name=$CODER_TEMPLATE_VERSION --message="$CODER_TEMPLATE_MESSAGE"
env:
+3
View File
@@ -17,6 +17,9 @@
},
{
"pattern": "tailscale.com"
},
{
"pattern": "wireguard.com"
}
],
"aliveStatusCodes": [200, 0]
+1 -1
View File
@@ -14,4 +14,4 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Assign author
uses: toshimaru/auto-author-assign@v2.0.1
uses: toshimaru/auto-author-assign@v2.1.0
+7 -10
View File
@@ -9,10 +9,6 @@ on:
- main
workflow_dispatch:
inputs:
pr_number:
description: "PR number"
type: number
required: true
experiments:
description: "Experiments to enable"
required: false
@@ -123,7 +119,7 @@ jobs:
echo "NEW=$NEW" >> $GITHUB_OUTPUT
- name: Check changed files
uses: dorny/paths-filter@v2
uses: dorny/paths-filter@v3
id: filter
with:
base: ${{ github.ref }}
@@ -167,7 +163,7 @@ jobs:
runs-on: "ubuntu-latest"
steps:
- name: Find Comment
uses: peter-evans/find-comment@v2
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ needs.get_info.outputs.PR_NUMBER }}
@@ -177,7 +173,7 @@ jobs:
- name: Comment on PR
id: comment_id
uses: peter-evans/create-or-update-comment@v3
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ needs.get_info.outputs.PR_NUMBER }}
@@ -355,6 +351,7 @@ jobs:
- name: Install/Upgrade Helm chart
run: |
set -euo pipefail
helm dependency update --skip-refresh ./helm/coder
helm upgrade --install "pr${{ env.PR_NUMBER }}" ./helm/coder \
--namespace "pr${{ env.PR_NUMBER }}" \
--values ./pr-deploy-values.yaml \
@@ -419,7 +416,7 @@ jobs:
# Create template
cd ./.github/pr-deployments/template
coder templates create -y --variable namespace=pr${{ env.PR_NUMBER }} kubernetes
coder templates push -y --variable namespace=pr${{ env.PR_NUMBER }} kubernetes
# Create workspace
coder create --template="kubernetes" kube --parameter cpu=2 --parameter memory=4 --parameter home_disk_size=2 -y
@@ -444,7 +441,7 @@ jobs:
echo "Slack notification sent"
- name: Find Comment
uses: peter-evans/find-comment@v2
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ env.PR_NUMBER }}
@@ -453,7 +450,7 @@ jobs:
direction: last
- name: Comment on PR
uses: peter-evans/create-or-update-comment@v3
uses: peter-evans/create-or-update-comment@v4
env:
STATUS: ${{ needs.get_info.outputs.NEW == 'true' && 'Created' || 'Updated' }}
with:
+115 -70
View File
@@ -1,11 +1,16 @@
# GitHub release workflow.
name: Release
on:
push:
tags:
- "v*"
workflow_dispatch:
inputs:
release_channel:
type: choice
description: Release channel
options:
- mainline
- stable
release_notes:
description: Release notes for the publishing the release. This is required to create a release.
dry_run:
description: Perform a dry-run release (devel). Note that ref must be an annotated tag when run without dry-run.
type: boolean
@@ -28,6 +33,8 @@ env:
# https://github.blog/changelog/2022-06-10-github-actions-inputs-unified-across-manual-and-reusable-workflows/
CODER_RELEASE: ${{ !inputs.dry_run }}
CODER_DRY_RUN: ${{ inputs.dry_run }}
CODER_RELEASE_CHANNEL: ${{ inputs.release_channel }}
CODER_RELEASE_NOTES: ${{ inputs.release_notes }}
jobs:
release:
@@ -62,21 +69,45 @@ jobs:
echo "CODER_FORCE_VERSION=$version" >> $GITHUB_ENV
echo "$version"
- name: Create release notes
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# We always have to set this since there might be commits on
# main that didn't have a PR.
CODER_IGNORE_MISSING_COMMIT_METADATA: "1"
# Verify that all expectations for a release are met.
- name: Verify release input
if: ${{ !inputs.dry_run }}
run: |
set -euo pipefail
if [[ "${GITHUB_REF}" != "refs/tags/v"* ]]; then
echo "Ref must be a semver tag when creating a release, did you use scripts/release.sh?"
exit 1
fi
# 2.10.2 -> release/2.10
version="$(./scripts/version.sh)"
release_branch=release/${version%.*}
branch_contains_tag=$(git branch --remotes --contains "${GITHUB_REF}" --list "*/${release_branch}" --format='%(refname)')
if [[ -z "${branch_contains_tag}" ]]; then
echo "Ref tag must exist in a branch named ${release_branch} when creating a release, did you use scripts/release.sh?"
exit 1
fi
if [[ -z "${CODER_RELEASE_NOTES}" ]]; then
echo "Release notes are required to create a release, did you use scripts/release.sh?"
exit 1
fi
echo "Release inputs verified:"
echo
echo "- Ref: ${GITHUB_REF}"
echo "- Version: ${version}"
echo "- Release channel: ${CODER_RELEASE_CHANNEL}"
echo "- Release branch: ${release_branch}"
echo "- Release notes: true"
- name: Create release notes file
run: |
set -euo pipefail
ref=HEAD
old_version="$(git describe --abbrev=0 "$ref^1")"
version="v$(./scripts/version.sh)"
# Generate notes.
release_notes_file="$(mktemp -t release_notes.XXXXXX)"
./scripts/release/generate_release_notes.sh --check-for-changelog --old-version "$old_version" --new-version "$version" --ref "$ref" >> "$release_notes_file"
echo "$CODER_RELEASE_NOTES" > "$release_notes_file"
echo CODER_RELEASE_NOTES_FILE="$release_notes_file" >> $GITHUB_ENV
- name: Show release notes
@@ -97,13 +128,20 @@ jobs:
- name: Setup Node
uses: ./.github/actions/setup-node
# Necessary for signing Windows binaries.
- name: Setup Java
uses: actions/setup-java@v4
with:
distribution: "zulu"
java-version: "11.0"
- name: Install nsis and zstd
run: sudo apt-get install -y nsis zstd
- name: Install nfpm
run: |
set -euo pipefail
wget -O /tmp/nfpm.deb https://github.com/goreleaser/nfpm/releases/download/v2.18.1/nfpm_amd64.deb
wget -O /tmp/nfpm.deb https://github.com/goreleaser/nfpm/releases/download/v2.35.1/nfpm_2.35.1_amd64.deb
sudo dpkg -i /tmp/nfpm.deb
rm /tmp/nfpm.deb
@@ -130,6 +168,32 @@ jobs:
AC_CERTIFICATE_PASSWORD: ${{ secrets.AC_CERTIFICATE_PASSWORD }}
AC_APIKEY_P8_BASE64: ${{ secrets.AC_APIKEY_P8_BASE64 }}
- name: Setup Windows EV Signing Certificate
run: |
set -euo pipefail
touch /tmp/ev_cert.pem
chmod 600 /tmp/ev_cert.pem
echo "$EV_SIGNING_CERT" > /tmp/ev_cert.pem
wget https://github.com/ebourg/jsign/releases/download/6.0/jsign-6.0.jar -O /tmp/jsign-6.0.jar
env:
EV_SIGNING_CERT: ${{ secrets.EV_SIGNING_CERT }}
# - name: Test migrations from current ref to main
# run: |
# make test-migrations
# Setup GCloud for signing Windows binaries.
- name: Authenticate to Google Cloud
id: gcloud_auth
uses: google-github-actions/auth@v2
with:
workload_identity_provider: ${{ secrets.GCP_CODE_SIGNING_WORKLOAD_ID_PROVIDER }}
service_account: ${{ secrets.GCP_CODE_SIGNING_SERVICE_ACCOUNT }}
token_format: "access_token"
- name: Setup GCloud SDK
uses: "google-github-actions/setup-gcloud@v2"
- name: Build binaries
run: |
set -euo pipefail
@@ -144,16 +208,26 @@ jobs:
build/coder_helm_"$version".tgz \
build/provisioner_helm_"$version".tgz
env:
CODER_SIGN_WINDOWS: "1"
CODER_SIGN_DARWIN: "1"
AC_CERTIFICATE_FILE: /tmp/apple_cert.p12
AC_CERTIFICATE_PASSWORD_FILE: /tmp/apple_cert_password.txt
AC_APIKEY_ISSUER_ID: ${{ secrets.AC_APIKEY_ISSUER_ID }}
AC_APIKEY_ID: ${{ secrets.AC_APIKEY_ID }}
AC_APIKEY_FILE: /tmp/apple_apikey.p8
EV_KEY: ${{ secrets.EV_KEY }}
EV_KEYSTORE: ${{ secrets.EV_KEYSTORE }}
EV_TSA_URL: ${{ secrets.EV_TSA_URL }}
EV_CERTIFICATE_PATH: /tmp/ev_cert.pem
GCLOUD_ACCESS_TOKEN: ${{ steps.gcloud_auth.outputs.access_token }}
JSIGN_PATH: /tmp/jsign-6.0.jar
- name: Delete Apple Developer certificate and API key
run: rm -f /tmp/{apple_cert.p12,apple_cert_password.txt,apple_apikey.p8}
- name: Delete Windows EV Signing Cert
run: rm /tmp/ev_cert.pem
- name: Determine base image tag
id: image-base-tag
run: |
@@ -261,6 +335,9 @@ jobs:
set -euo pipefail
publish_args=()
if [[ $CODER_RELEASE_CHANNEL == "stable" ]]; then
publish_args+=(--stable)
fi
if [[ $CODER_DRY_RUN == *t* ]]; then
publish_args+=(--dry-run)
fi
@@ -306,7 +383,7 @@ jobs:
- name: Upload artifacts to actions (if dry-run)
if: ${{ inputs.dry_run }}
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: release-artifacts
path: |
@@ -321,7 +398,7 @@ jobs:
- name: Start Packer builds
if: ${{ !inputs.dry_run }}
uses: peter-evans/repository-dispatch@v2
uses: peter-evans/repository-dispatch@v3
with:
token: ${{ secrets.CDRCI_GITHUB_TOKEN }}
repository: coder/packages
@@ -408,6 +485,11 @@ jobs:
if: ${{ !inputs.dry_run }}
steps:
- name: Sync fork
run: gh repo sync cdrci/winget-pkgs -b master
env:
GH_TOKEN: ${{ secrets.CDRCI_GITHUB_TOKEN }}
- name: Checkout
uses: actions/checkout@v4
with:
@@ -480,65 +562,28 @@ jobs:
# different repo.
GH_TOKEN: ${{ secrets.CDRCI_GITHUB_TOKEN }}
publish-chocolatey:
name: Publish to Chocolatey
runs-on: windows-latest
# publish-sqlc pushes the latest schema to sqlc cloud.
# At present these pushes cannot be tagged, so the last push is always the latest.
publish-sqlc:
name: "Publish to schema sqlc cloud"
runs-on: "ubuntu-latest"
needs: release
if: ${{ !inputs.dry_run }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-depth: 1
# Same reason as for release.
- name: Fetch git tags
run: git fetch --tags --force
# We need golang to run the migration main.go
- name: Setup Go
uses: ./.github/actions/setup-go
# From https://chocolatey.org
- name: Install Chocolatey
- name: Setup sqlc
uses: ./.github/actions/setup-sqlc
- name: Push schema to sqlc cloud
# Don't block a release on this
continue-on-error: true
run: |
Set-ExecutionPolicy Bypass -Scope Process -Force
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072
iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
- name: Build chocolatey package
run: |
cd scripts/chocolatey
# The package version is the same as the tag minus the leading "v".
# The version in this output already has the leading "v" removed but
# we do it again to be safe.
$version = "${{ needs.release.outputs.version }}".Trim('v')
$release_assets = gh release view --repo coder/coder "v${version}" --json assets | `
ConvertFrom-Json
# Get the URL for the Windows ZIP from the release assets.
$zip_url = $release_assets.assets | `
Where-Object name -Match ".*_windows_amd64.zip$" | `
Select -ExpandProperty url
echo "ZIP URL: ${zip_url}"
echo "Package version: ${version}"
echo "Downloading ZIP..."
Invoke-WebRequest $zip_url -OutFile assets.zip
echo "Extracting ZIP..."
Expand-Archive assets.zip -DestinationPath assets/
# No need to specify nuspec if there's only one in the directory.
choco pack --version=$version binary_path=assets/coder.exe
choco apikey --api-key $env:CHOCO_API_KEY --source https://push.chocolatey.org/
# No need to specify nupkg if there's only one in the directory.
choco push --source https://push.chocolatey.org/
env:
CHOCO_API_KEY: ${{ secrets.CHOCO_API_KEY }}
# We need a GitHub token for the gh CLI to function under GitHub Actions
GH_TOKEN: ${{ secrets.CDRCI_GITHUB_TOKEN }}
make sqlc-push
+20 -18
View File
@@ -28,21 +28,21 @@ jobs:
- name: Checkout
uses: actions/checkout@v4
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: go, javascript
- name: Setup Go
uses: ./.github/actions/setup-go
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: go, javascript
# Workaround to prevent CodeQL from building the dashboard.
- name: Remove Makefile
run: |
rm Makefile
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
uses: github/codeql-action/analyze@v3
- name: Send Slack notification on failure
if: ${{ failure() }}
@@ -75,7 +75,7 @@ jobs:
- name: Install yq
run: go run github.com/mikefarah/yq/v4@v4.30.6
- name: Install mockgen
run: go install github.com/golang/mock/mockgen@v1.6.0
run: go install go.uber.org/mock/mockgen@v0.4.0
- name: Install protoc-gen-go
run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.30
- name: Install protoc-gen-go-drpc
@@ -113,16 +113,8 @@ jobs:
make -j "$image_job"
echo "image=$(cat "$image_job")" >> $GITHUB_OUTPUT
- name: Run Prisma Cloud image scan
uses: PaloAltoNetworks/prisma-cloud-scan@v1
with:
pcc_console_url: ${{ secrets.PRISMA_CLOUD_URL }}
pcc_user: ${{ secrets.PRISMA_CLOUD_ACCESS_KEY }}
pcc_pass: ${{ secrets.PRISMA_CLOUD_SECRET_KEY }}
image_name: ${{ steps.build.outputs.image }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@91713af97dc80187565512baba96e4364e983601
uses: aquasecurity/trivy-action@d710430a6722f083d3b36b8339ff66b32f22ee55
with:
image-ref: ${{ steps.build.outputs.image }}
format: sarif
@@ -130,18 +122,28 @@ jobs:
severity: "CRITICAL,HIGH"
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v2
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: trivy-results.sarif
category: "Trivy"
- name: Upload Trivy scan results as an artifact
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: trivy
path: trivy-results.sarif
retention-days: 7
# Prisma cloud scan runs last because it fails the entire job if it
# detects vulnerabilities. :|
- name: Run Prisma Cloud image scan
uses: PaloAltoNetworks/prisma-cloud-scan@v1
with:
pcc_console_url: ${{ secrets.PRISMA_CLOUD_URL }}
pcc_user: ${{ secrets.PRISMA_CLOUD_ACCESS_KEY }}
pcc_pass: ${{ secrets.PRISMA_CLOUD_SECRET_KEY }}
image_name: ${{ steps.build.outputs.image }}
- name: Send Slack notification on failure
if: ${{ failure() }}
run: |
+1 -1
View File
@@ -68,7 +68,7 @@ jobs:
repo: context.repo.repo,
issue_number: issue.number,
state: 'closed',
state_reason: 'not planned'
state_reason: 'not_planned'
});
}
} else {
+4 -1
View File
@@ -14,7 +14,8 @@ darcula = "darcula"
Hashi = "Hashi"
trialer = "trialer"
encrypter = "encrypter"
hel = "hel" # as in helsinki
hel = "hel" # as in helsinki
pn = "pn" # this is used as proto node
[files]
extend-exclude = [
@@ -30,4 +31,6 @@ extend-exclude = [
"**/*_test.go",
"**/*.test.tsx",
"**/pnpm-lock.yaml",
"tailnet/testdata/**",
"site/src/pages/SetupPage/countries.tsx",
]
+6 -1
View File
@@ -4,6 +4,11 @@ on:
schedule:
- cron: "0 9 * * 1"
workflow_dispatch: # allows to run manually for testing
pull_request:
branches:
- main
paths:
- "docs/**"
jobs:
check-docs:
@@ -24,7 +29,7 @@ jobs:
file-path: "./README.md"
- name: Send Slack notification
if: failure()
if: failure() && github.event_name != 'workflow_dispatch'
run: |
curl -X POST -H 'Content-type: application/json' -d '{"msg":"Broken links found in the documentation. Please check the logs at ${{ env.LOGS_URL }}"}' ${{ secrets.DOCS_LINK_SLACK_WEBHOOK }}
echo "Sent Slack notification"
+6 -3
View File
@@ -21,8 +21,8 @@
"contravariance",
"cronstrue",
"databasefake",
"dbmem",
"dbgen",
"dbmem",
"dbtype",
"DERP",
"derphttp",
@@ -60,6 +60,7 @@
"idtoken",
"Iflag",
"incpatch",
"initialisms",
"ipnstate",
"isatty",
"Jobf",
@@ -113,18 +114,19 @@
"Signup",
"slogtest",
"sourcemapped",
"spinbutton",
"Srcs",
"stdbuf",
"stretchr",
"STTY",
"stuntest",
"tanstack",
"tailbroker",
"tailcfg",
"tailexchange",
"tailnet",
"tailnettest",
"Tailscale",
"tanstack",
"tbody",
"TCGETS",
"tcpip",
@@ -141,6 +143,7 @@
"tios",
"tmpdir",
"tokenconfig",
"Topbar",
"tparallel",
"trialer",
"trimprefix",
@@ -168,10 +171,10 @@
"workspaceapps",
"workspacebuilds",
"workspacename",
"wsconncache",
"wsjson",
"xerrors",
"xlarge",
"xsmall",
"yamux"
],
"cSpell.ignorePaths": ["site/package.json", ".vscode/settings.json"],
+87 -15
View File
@@ -200,7 +200,8 @@ endef
# calling this manually.
$(CODER_ALL_BINARIES): go.mod go.sum \
$(GO_SRC_FILES) \
$(shell find ./examples/templates)
$(shell find ./examples/templates) \
site/static/error.html
$(get-mode-os-arch-ext)
if [[ "$$os" != "windows" ]] && [[ "$$ext" != "" ]]; then
@@ -361,6 +362,8 @@ $(foreach chart,$(charts),build/$(chart)_helm_$(VERSION).tgz): build/%_helm_$(VE
site/out/index.html: site/package.json $(shell find ./site $(FIND_EXCLUSIONS) -type f \( -name '*.ts' -o -name '*.tsx' \))
cd site
# prevents this directory from getting to big, and causing "too much data" errors
rm -rf out/assets/
../scripts/pnpm_install.sh
pnpm build
@@ -380,32 +383,44 @@ install: build/coder_$(VERSION)_$(GOOS)_$(GOARCH)$(GOOS_BIN_EXT)
cp "$<" "$$output_file"
.PHONY: install
fmt: fmt/prettier fmt/terraform fmt/shfmt fmt/go
BOLD := $(shell tput bold 2>/dev/null)
GREEN := $(shell tput setaf 2 2>/dev/null)
RESET := $(shell tput sgr0 2>/dev/null)
fmt: fmt/eslint fmt/prettier fmt/terraform fmt/shfmt fmt/go
.PHONY: fmt
fmt/go:
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/go$(RESET)"
# VS Code users should check out
# https://github.com/mvdan/gofumpt#visual-studio-code
go run mvdan.cc/gofumpt@v0.4.0 -w -l .
.PHONY: fmt/go
fmt/eslint:
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/eslint$(RESET)"
cd site
pnpm run lint:fix
.PHONY: fmt/eslint
fmt/prettier:
echo "--- prettier"
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/prettier$(RESET)"
cd site
# Avoid writing files in CI to reduce file write activity
ifdef CI
pnpm run format:check
else
pnpm run format:write
pnpm run format
endif
.PHONY: fmt/prettier
fmt/terraform: $(wildcard *.tf)
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/terraform$(RESET)"
terraform fmt -recursive
.PHONY: fmt/terraform
fmt/shfmt: $(SHELL_SRC_FILES)
echo "--- shfmt"
echo "$(GREEN)==>$(RESET) $(BOLD)fmt/shfmt$(RESET)"
# Only do diff check in CI, errors on diff.
ifdef CI
shfmt -d $(SHELL_SRC_FILES)
@@ -414,7 +429,7 @@ else
endif
.PHONY: fmt/shfmt
lint: lint/shellcheck lint/go lint/ts lint/helm lint/site-icons
lint: lint/shellcheck lint/go lint/ts lint/examples lint/helm lint/site-icons
.PHONY: lint
lint/site-icons:
@@ -433,6 +448,10 @@ lint/go:
golangci-lint run
.PHONY: lint/go
lint/examples:
go run ./scripts/examplegen/main.go -lint
.PHONY: lint/examples
# Use shfmt to determine the shell files, takes editorconfig into consideration.
lint/shellcheck: $(SHELL_SRC_FILES)
echo "--- shellcheck"
@@ -470,12 +489,16 @@ gen: \
coderd/apidoc/swagger.json \
.prettierignore.include \
.prettierignore \
provisioner/terraform/testdata/version \
site/.prettierrc.yaml \
site/.prettierignore \
site/.eslintignore \
site/e2e/provisionerGenerated.ts \
site/src/theme/icons.json \
examples/examples.gen.json
examples/examples.gen.json \
tailnet/tailnettest/coordinatormock.go \
tailnet/tailnettest/coordinateemock.go \
tailnet/tailnettest/multiagentmock.go
.PHONY: gen
# Mark all generated files as fresh so make thinks they're up-to-date. This is
@@ -502,6 +525,9 @@ gen/mark-fresh:
site/e2e/provisionerGenerated.ts \
site/src/theme/icons.json \
examples/examples.gen.json \
tailnet/tailnettest/coordinatormock.go \
tailnet/tailnettest/coordinateemock.go \
tailnet/tailnettest/multiagentmock.go \
"
for file in $$files; do
echo "$$file"
@@ -529,6 +555,9 @@ coderd/database/querier.go: coderd/database/sqlc.yaml coderd/database/dump.sql $
coderd/database/dbmock/dbmock.go: coderd/database/db.go coderd/database/querier.go
go generate ./coderd/database/dbmock/
tailnet/tailnettest/coordinatormock.go tailnet/tailnettest/multiagentmock.go tailnet/tailnettest/coordinateemock.go: tailnet/coordinator.go tailnet/multiagent.go
go generate ./tailnet/tailnettest/
tailnet/proto/tailnet.pb.go: tailnet/proto/tailnet.proto
protoc \
--go_out=. \
@@ -563,7 +592,8 @@ provisionerd/proto/provisionerd.pb.go: provisionerd/proto/provisionerd.proto
site/src/api/typesGenerated.ts: $(wildcard scripts/apitypings/*) $(shell find ./codersdk $(FIND_EXCLUSIONS) -type f -name '*.go')
go run ./scripts/apitypings/ > $@
pnpm run format:write:only "$@"
./scripts/pnpm_install.sh
pnpm exec prettier --write "$@"
site/e2e/provisionerGenerated.ts: provisionerd/proto/provisionerd.pb.go provisionersdk/proto/provisioner.pb.go
cd site
@@ -572,7 +602,8 @@ site/e2e/provisionerGenerated.ts: provisionerd/proto/provisionerd.pb.go provisio
site/src/theme/icons.json: $(wildcard scripts/gensite/*) $(wildcard site/static/icon/*)
go run ./scripts/gensite/ -icons "$@"
pnpm run format:write:only "$@"
./scripts/pnpm_install.sh
pnpm exec prettier --write "$@"
examples/examples.gen.json: scripts/examplegen/main.go examples/examples.go $(shell find ./examples/templates)
go run ./scripts/examplegen/main.go > examples/examples.gen.json
@@ -582,19 +613,23 @@ coderd/rbac/object_gen.go: scripts/rbacgen/main.go coderd/rbac/object.go
docs/admin/prometheus.md: scripts/metricsdocgen/main.go scripts/metricsdocgen/metrics
go run scripts/metricsdocgen/main.go
pnpm run format:write:only ./docs/admin/prometheus.md
./scripts/pnpm_install.sh
pnpm exec prettier --write ./docs/admin/prometheus.md
docs/cli.md: scripts/clidocgen/main.go examples/examples.gen.json $(GO_SRC_FILES)
CI=true BASE_PATH="." go run ./scripts/clidocgen
pnpm run format:write:only ./docs/cli.md ./docs/cli/*.md ./docs/manifest.json
./scripts/pnpm_install.sh
pnpm exec prettier --write ./docs/cli.md ./docs/cli/*.md ./docs/manifest.json
docs/admin/audit-logs.md: scripts/auditdocgen/main.go enterprise/audit/table.go coderd/rbac/object_gen.go
docs/admin/audit-logs.md: coderd/database/querier.go scripts/auditdocgen/main.go enterprise/audit/table.go coderd/rbac/object_gen.go
go run scripts/auditdocgen/main.go
pnpm run format:write:only ./docs/admin/audit-logs.md
./scripts/pnpm_install.sh
pnpm exec prettier --write ./docs/admin/audit-logs.md
coderd/apidoc/swagger.json: $(shell find ./scripts/apidocgen $(FIND_EXCLUSIONS) -type f) $(wildcard coderd/*.go) $(wildcard enterprise/coderd/*.go) $(wildcard codersdk/*.go) $(wildcard enterprise/wsproxy/wsproxysdk/*.go) $(DB_GEN_FILES) .swaggo docs/manifest.json coderd/rbac/object_gen.go
./scripts/apidocgen/generate.sh
pnpm run format:write:only ./docs/api ./docs/manifest.json ./coderd/apidoc/swagger.json
./scripts/pnpm_install.sh
pnpm exec prettier --write ./docs/api ./docs/manifest.json ./coderd/apidoc/swagger.json
update-golden-files: \
cli/testdata/.gen-golden \
@@ -609,7 +644,7 @@ update-golden-files: \
.PHONY: update-golden-files
cli/testdata/.gen-golden: $(wildcard cli/testdata/*.golden) $(wildcard cli/*.tpl) $(GO_SRC_FILES) $(wildcard cli/*_test.go)
go test ./cli -run="Test(CommandHelp|ServerYAML)" -update
go test ./cli -run="Test(CommandHelp|ServerYAML|ErrorExamples)" -update
touch "$@"
enterprise/cli/testdata/.gen-golden: $(wildcard enterprise/cli/testdata/*.golden) $(wildcard cli/*.tpl) $(GO_SRC_FILES) $(wildcard enterprise/cli/*_test.go)
@@ -640,6 +675,12 @@ provisioner/terraform/testdata/.gen-golden: $(wildcard provisioner/terraform/tes
go test ./provisioner/terraform -run="Test.*Golden$$" -update
touch "$@"
provisioner/terraform/testdata/version:
if [[ "$(shell cat provisioner/terraform/testdata/version.txt)" != "$(shell terraform version -json | jq -r '.terraform_version')" ]]; then
./provisioner/terraform/testdata/generate.sh
fi
.PHONY: provisioner/terraform/testdata/version
scripts/ci-report/testdata/.gen-golden: $(wildcard scripts/ci-report/testdata/*) $(wildcard scripts/ci-report/*.go)
go test ./scripts/ci-report -run=TestOutputMatchesGoldenFile -update
touch "$@"
@@ -708,6 +749,27 @@ test:
gotestsum --format standard-quiet -- -v -short -count=1 ./...
.PHONY: test
# sqlc-cloud-is-setup will fail if no SQLc auth token is set. Use this as a
# dependency for any sqlc-cloud related targets.
sqlc-cloud-is-setup:
if [[ "$(SQLC_AUTH_TOKEN)" == "" ]]; then
echo "ERROR: 'SQLC_AUTH_TOKEN' must be set to auth with sqlc cloud before running verify." 1>&2
exit 1
fi
.PHONY: sqlc-cloud-is-setup
sqlc-push: sqlc-cloud-is-setup test-postgres-docker
echo "--- sqlc push"
SQLC_DATABASE_URL="postgresql://postgres:postgres@localhost:5432/$(shell go run scripts/migrate-ci/main.go)" \
sqlc push -f coderd/database/sqlc.yaml && echo "Passed sqlc push"
.PHONY: sqlc-push
sqlc-verify: sqlc-cloud-is-setup test-postgres-docker
echo "--- sqlc verify"
SQLC_DATABASE_URL="postgresql://postgres:postgres@localhost:5432/$(shell go run scripts/migrate-ci/main.go)" \
sqlc verify -f coderd/database/sqlc.yaml && echo "Passed sqlc verify"
.PHONY: sqlc-verify
sqlc-vet: test-postgres-docker
echo "--- sqlc vet"
SQLC_DATABASE_URL="postgresql://postgres:postgres@localhost:5432/$(shell go run scripts/migrate-ci/main.go)" \
@@ -729,6 +791,15 @@ test-postgres: test-postgres-docker
-count=1
.PHONY: test-postgres
test-migrations: test-postgres-docker
echo "--- test migrations"
set -euo pipefail
COMMIT_FROM=$(shell git rev-parse --short HEAD)
COMMIT_TO=$(shell git rev-parse --short main)
echo "DROP DATABASE IF EXISTS migrate_test_$${COMMIT_FROM}; CREATE DATABASE migrate_test_$${COMMIT_FROM};" | psql 'postgresql://postgres:postgres@localhost:5432/postgres?sslmode=disable'
go run ./scripts/migrate-test/main.go --from="$$COMMIT_FROM" --to="$$COMMIT_TO" --postgres-url="postgresql://postgres:postgres@localhost:5432/migrate_test_$${COMMIT_FROM}?sslmode=disable"
# NOTE: we set --memory to the same size as a GitHub runner.
test-postgres-docker:
docker rm -f test-postgres-docker || true
docker run \
@@ -741,6 +812,7 @@ test-postgres-docker:
--name test-postgres-docker \
--restart no \
--detach \
--memory 16GB \
gcr.io/coder-dev-1/postgres:13 \
-c shared_buffers=1GB \
-c work_mem=1GB \
+20 -21
View File
@@ -7,7 +7,7 @@
</a>
<h1>
Self-Hosted Remote Development Environments
Self-Hosted Cloud Development Environments
</h1>
<a href="https://coder.com#gh-light-mode-only">
@@ -23,7 +23,6 @@
[Quickstart](#quickstart) | [Docs](https://coder.com/docs) | [Why Coder](https://coder.com/why) | [Enterprise](https://coder.com/docs/v2/latest/enterprise)
[![discord](https://img.shields.io/discord/747933592273027093?label=discord)](https://discord.gg/coder)
[![codecov](https://codecov.io/gh/coder/coder/branch/main/graph/badge.svg?token=TNLW3OAP6G)](https://codecov.io/gh/coder/coder)
[![release](https://img.shields.io/github/v/release/coder/coder)](https://github.com/coder/coder/releases/latest)
[![godoc](https://pkg.go.dev/badge/github.com/coder/coder.svg)](https://pkg.go.dev/github.com/coder/coder)
[![Go Report Card](https://goreportcard.com/badge/github.com/coder/coder)](https://goreportcard.com/report/github.com/coder/coder)
@@ -31,9 +30,9 @@
</div>
[Coder](https://coder.com) enables organizations to set up development environments in the cloud. Environments are defined with Terraform, connected through a secure high-speed Wireguard® tunnel, and are automatically shut down when not in use to save on costs. Coder gives engineering teams the flexibility to use the cloud for workloads that are most beneficial to them.
[Coder](https://coder.com) enables organizations to set up development environments in their public or private cloud infrastructure. Cloud development environments are defined with Terraform, connected through a secure high-speed Wireguard® tunnel, and are automatically shut down when not in use to save on costs. Coder gives engineering teams the flexibility to use the cloud for workloads that are most beneficial to them.
- Define development environments in Terraform
- Define cloud development environments in Terraform
- EC2 VMs, Kubernetes Pods, Docker Containers, etc.
- Automatically shutdown idle resources to save on costs
- Onboard developers in seconds instead of days
@@ -44,7 +43,7 @@
## Quickstart
The most convenient way to try Coder is to install it on your local machine and experiment with provisioning development environments using Docker (works on Linux, macOS, and Windows).
The most convenient way to try Coder is to install it on your local machine and experiment with provisioning cloud development environments using Docker (works on Linux, macOS, and Windows).
```
# First, install Coder
@@ -53,8 +52,8 @@ curl -L https://coder.com/install.sh | sh
# Start the Coder server (caches data in ~/.cache/coder)
coder server
# Navigate to http://localhost:3000 to create your initial user
# Create a Docker template, and provision a workspace
# Navigate to http://localhost:3000 to create your initial user,
# create a Docker template, and provision a workspace
```
## Install
@@ -68,11 +67,11 @@ Releases.
curl -L https://coder.com/install.sh | sh
```
You can run the install script with `--dry-run` to see the commands that will be used to install without executing them. You can modify the installation process by including flags. Run the install script with `--help` for reference.
You can run the install script with `--dry-run` to see the commands that will be used to install without executing them. Run the install script with `--help` for additional flags.
> See [install](https://coder.com/docs/v2/latest/install) for additional methods.
Once installed, you can start a production deployment<sup>1</sup> with a single command:
Once installed, you can start a production deployment with a single command:
```shell
# Automatically sets up an external access URL on *.try.coder.app
@@ -82,8 +81,6 @@ coder server
coder server --postgres-url <url> --access-url <url>
```
> <sup>1</sup> For production deployments, set up an external PostgreSQL instance for reliability.
Use `coder --help` to get a list of flags and environment variables. Use our [install guides](https://coder.com/docs/v2/latest/install) for a full walkthrough.
## Documentation
@@ -96,19 +93,13 @@ Browse our docs [here](https://coder.com/docs/v2) or visit a specific section be
- [**Administration**](https://coder.com/docs/v2/latest/admin): Learn how to operate Coder
- [**Enterprise**](https://coder.com/docs/v2/latest/enterprise): Learn about our paid features built for large teams
## Community and Support
## Support
Feel free to [open an issue](https://github.com/coder/coder/issues/new) if you have questions, run into bugs, or have a feature request.
[Join our Discord](https://discord.gg/coder) to provide feedback on in-progress features, and chat with the community using Coder!
## Contributing
Contributions are welcome! Read the [contributing docs](https://coder.com/docs/v2/latest/CONTRIBUTING) to get started.
Find our list of contributors [here](https://github.com/coder/coder/graphs/contributors).
## Related
## Integrations
We are always working on new integrations. Feel free to open an issue to request an integration. Contributions are welcome in any official or community repositories.
@@ -116,10 +107,18 @@ We are always working on new integrations. Feel free to open an issue to request
- [**VS Code Extension**](https://marketplace.visualstudio.com/items?itemName=coder.coder-remote): Open any Coder workspace in VS Code with a single click
- [**JetBrains Gateway Extension**](https://plugins.jetbrains.com/plugin/19620-coder): Open any Coder workspace in JetBrains Gateway with a single click
- [**Dev Container Builder**](https://github.com/coder/envbuilder): Build development environments using `devcontainer.json` on Docker, Kubernetes, and OpenShift
- [**Module Registry**](https://registry.coder.com): Extend development environments with common use-cases
- [**Kubernetes Log Stream**](https://github.com/coder/coder-logstream-kube): Stream Kubernetes Pod events to the Coder startup logs
- [**Self-Hosted VS Code Extension Marketplace**](https://github.com/coder/code-marketplace): A private extension marketplace that works in restricted or airgapped networks integrating with [code-server](https://github.com/coder/code-server).
### Community
- [**Provision Coder with Terraform**](https://github.com/ElliotG/coder-oss-tf): Provision Coder on Google GKE, Azure AKS, AWS EKS, DigitalOcean DOKS, IBMCloud K8s, OVHCloud K8s, and Scaleway K8s Kapsule with Terraform
- [**Coder GitHub Action**](https://github.com/marketplace/actions/update-coder-template): A GitHub Action that updates Coder templates
- [**Various Templates**](./examples/templates/community-templates.md): Hetzner Cloud, Docker in Docker, and other templates the community has built.
- [**Coder Template GitHub Action**](https://github.com/marketplace/actions/update-coder-template): A GitHub Action that updates Coder templates
## Contributing
We are always happy to see new contributors to Coder. If you are new to the Coder codebase, we have
[a guide on how to get started](https://coder.com/docs/v2/latest/CONTRIBUTING). We'd love to see your
contributions!
+1050 -496
View File
File diff suppressed because it is too large Load Diff
+421 -125
View File
@@ -5,9 +5,9 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"math/rand"
"net"
"net/http"
"net/http/httptest"
@@ -27,7 +27,6 @@ import (
"time"
"github.com/bramvdbogaerde/go-scp"
"github.com/golang/mock/gomock"
"github.com/google/uuid"
"github.com/pion/udp"
"github.com/pkg/sftp"
@@ -37,6 +36,7 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/goleak"
"go.uber.org/mock/gomock"
"golang.org/x/crypto/ssh"
"golang.org/x/exp/slices"
"golang.org/x/xerrors"
@@ -46,14 +46,16 @@ import (
"cdr.dev/slog"
"cdr.dev/slog/sloggers/sloghuman"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent"
"github.com/coder/coder/v2/agent/agentproc"
"github.com/coder/coder/v2/agent/agentproc/agentproctest"
"github.com/coder/coder/v2/agent/agentssh"
"github.com/coder/coder/v2/agent/agenttest"
"github.com/coder/coder/v2/agent/proto"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
"github.com/coder/coder/v2/cryptorand"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/coder/v2/tailnet"
"github.com/coder/coder/v2/tailnet/tailnettest"
@@ -85,11 +87,11 @@ func TestAgent_Stats_SSH(t *testing.T) {
err = session.Shell()
require.NoError(t, err)
var s *agentsdk.Stats
var s *proto.Stats
require.Eventuallyf(t, func() bool {
var ok bool
s, ok = <-stats
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 && s.SessionCountSSH == 1
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 && s.SessionCountSsh == 1
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats: %+v", s,
)
@@ -111,18 +113,18 @@ func TestAgent_Stats_ReconnectingPTY(t *testing.T) {
require.NoError(t, err)
defer ptyConn.Close()
data, err := json.Marshal(codersdk.ReconnectingPTYRequest{
data, err := json.Marshal(workspacesdk.ReconnectingPTYRequest{
Data: "echo test\r\n",
})
require.NoError(t, err)
_, err = ptyConn.Write(data)
require.NoError(t, err)
var s *agentsdk.Stats
var s *proto.Stats
require.Eventuallyf(t, func() bool {
var ok bool
s, ok = <-stats
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 && s.SessionCountReconnectingPTY == 1
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 && s.SessionCountReconnectingPty == 1
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats: %+v", s,
)
@@ -174,19 +176,19 @@ func TestAgent_Stats_Magic(t *testing.T) {
require.NoError(t, err)
err = session.Shell()
require.NoError(t, err)
var s *agentsdk.Stats
require.Eventuallyf(t, func() bool {
var ok bool
s, ok = <-stats
s, ok := <-stats
t.Logf("got stats: ok=%t, ConnectionCount=%d, RxBytes=%d, TxBytes=%d, SessionCountVSCode=%d, ConnectionMedianLatencyMS=%f",
ok, s.ConnectionCount, s.RxBytes, s.TxBytes, s.SessionCountVscode, s.ConnectionMedianLatencyMs)
return ok && s.ConnectionCount > 0 && s.RxBytes > 0 && s.TxBytes > 0 &&
// Ensure that the connection didn't count as a "normal" SSH session.
// This was a special one, so it should be labeled specially in the stats!
s.SessionCountVSCode == 1 &&
s.SessionCountVscode == 1 &&
// Ensure that connection latency is being counted!
// If it isn't, it's set to -1.
s.ConnectionMedianLatencyMS >= 0
s.ConnectionMedianLatencyMs >= 0
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats: %+v", s,
"never saw stats",
)
// The shell will automatically exit if there is no stdin!
_ = stdin.Close()
@@ -240,14 +242,14 @@ func TestAgent_Stats_Magic(t *testing.T) {
_ = tunneledConn.Close()
})
var s *agentsdk.Stats
require.Eventuallyf(t, func() bool {
var ok bool
s, ok = <-stats
s, ok := <-stats
t.Logf("got stats with conn open: ok=%t, ConnectionCount=%d, SessionCountJetBrains=%d",
ok, s.ConnectionCount, s.SessionCountJetbrains)
return ok && s.ConnectionCount > 0 &&
s.SessionCountJetBrains == 1
s.SessionCountJetbrains == 1
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats with conn open: %+v", s,
"never saw stats with conn open",
)
// Kill the server and connection after checking for the echo.
@@ -256,12 +258,13 @@ func TestAgent_Stats_Magic(t *testing.T) {
_ = tunneledConn.Close()
require.Eventuallyf(t, func() bool {
var ok bool
s, ok = <-stats
return ok && s.ConnectionCount == 0 &&
s.SessionCountJetBrains == 0
s, ok := <-stats
t.Logf("got stats after disconnect %t, %d",
ok, s.SessionCountJetbrains)
return ok &&
s.SessionCountJetbrains == 0
}, testutil.WaitLong, testutil.IntervalFast,
"never saw stats after conn closes: %+v", s,
"never saw stats after conn closes",
)
})
}
@@ -279,6 +282,91 @@ func TestAgent_SessionExec(t *testing.T) {
require.Equal(t, "test", strings.TrimSpace(string(output)))
}
//nolint:tparallel // Sub tests need to run sequentially.
func TestAgent_Session_EnvironmentVariables(t *testing.T) {
t.Parallel()
tmpdir := t.TempDir()
// Defined by the coder script runner, hardcoded here since we don't
// have a reference to it.
scriptBinDir := filepath.Join(tmpdir, "coder-script-data", "bin")
manifest := agentsdk.Manifest{
EnvironmentVariables: map[string]string{
"MY_MANIFEST": "true",
"MY_OVERRIDE": "false",
"MY_SESSION_MANIFEST": "false",
},
}
banner := codersdk.ServiceBannerConfig{}
session := setupSSHSession(t, manifest, banner, nil, func(_ *agenttest.Client, opts *agent.Options) {
opts.ScriptDataDir = tmpdir
opts.EnvironmentVariables["MY_OVERRIDE"] = "true"
})
err := session.Setenv("MY_SESSION_MANIFEST", "true")
require.NoError(t, err)
err = session.Setenv("MY_SESSION", "true")
require.NoError(t, err)
command := "sh"
echoEnv := func(t *testing.T, w io.Writer, env string) {
if runtime.GOOS == "windows" {
_, err := fmt.Fprintf(w, "echo %%%s%%\r\n", env)
require.NoError(t, err)
} else {
_, err := fmt.Fprintf(w, "echo $%s\n", env)
require.NoError(t, err)
}
}
if runtime.GOOS == "windows" {
command = "cmd.exe"
}
stdin, err := session.StdinPipe()
require.NoError(t, err)
defer stdin.Close()
stdout, err := session.StdoutPipe()
require.NoError(t, err)
err = session.Start(command)
require.NoError(t, err)
// Context is fine here since we're not doing a parallel subtest.
ctx := testutil.Context(t, testutil.WaitLong)
go func() {
<-ctx.Done()
_ = session.Close()
}()
s := bufio.NewScanner(stdout)
//nolint:paralleltest // These tests need to run sequentially.
for k, partialV := range map[string]string{
"CODER": "true", // From the agent.
"MY_MANIFEST": "true", // From the manifest.
"MY_OVERRIDE": "true", // From the agent environment variables option, overrides manifest.
"MY_SESSION_MANIFEST": "false", // From the manifest, overrides session env.
"MY_SESSION": "true", // From the session.
"PATH": scriptBinDir + string(filepath.ListSeparator),
} {
t.Run(k, func(t *testing.T) {
echoEnv(t, stdin, k)
// Windows is unreliable, so keep scanning until we find a match.
for s.Scan() {
got := strings.TrimSpace(s.Text())
t.Logf("%s=%s", k, got)
if strings.Contains(got, partialV) {
break
}
}
if err := s.Err(); !errors.Is(err, io.EOF) {
require.NoError(t, err)
}
})
}
}
func TestAgent_GitSSH(t *testing.T) {
t.Parallel()
session := setupSSHSession(t, agentsdk.Manifest{}, codersdk.ServiceBannerConfig{}, nil)
@@ -750,7 +838,7 @@ func TestAgent_TCPRemoteForwarding(t *testing.T) {
var ll net.Listener
var err error
for {
randomPort = pickRandomPort()
randomPort = testutil.RandomPortNoListen(t)
addr := net.TCPAddrFromAddrPort(netip.AddrPortFrom(localhost, randomPort))
ll, err = sshClient.ListenTCP(addr)
if err != nil {
@@ -925,7 +1013,7 @@ func TestAgent_EnvironmentVariableExpansion(t *testing.T) {
func TestAgent_CoderEnvVars(t *testing.T) {
t.Parallel()
for _, key := range []string{"CODER"} {
for _, key := range []string{"CODER", "CODER_WORKSPACE_NAME", "CODER_WORKSPACE_AGENT_NAME"} {
key := key
t.Run(key, func(t *testing.T) {
t.Parallel()
@@ -1345,9 +1433,10 @@ func TestAgent_Lifecycle(t *testing.T) {
RunOnStop: true,
}},
},
make(chan *agentsdk.Stats, 50),
make(chan *proto.Stats, 50),
tailnet.NewCoordinator(logger),
)
defer client.Close()
fs := afero.NewMemMapFs()
agent := agent.New(agent.Options{
@@ -1392,56 +1481,52 @@ func TestAgent_Startup(t *testing.T) {
t.Run("EmptyDirectory", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Directory: "",
}, 0)
assert.Eventually(t, func() bool {
return client.GetStartup().Version != ""
}, testutil.WaitShort, testutil.IntervalFast)
require.Equal(t, "", client.GetStartup().ExpandedDirectory)
startup := testutil.RequireRecvCtx(ctx, t, client.GetStartup())
require.Equal(t, "", startup.GetExpandedDirectory())
})
t.Run("HomeDirectory", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Directory: "~",
}, 0)
assert.Eventually(t, func() bool {
return client.GetStartup().Version != ""
}, testutil.WaitShort, testutil.IntervalFast)
startup := testutil.RequireRecvCtx(ctx, t, client.GetStartup())
homeDir, err := os.UserHomeDir()
require.NoError(t, err)
require.Equal(t, homeDir, client.GetStartup().ExpandedDirectory)
require.Equal(t, homeDir, startup.GetExpandedDirectory())
})
t.Run("NotAbsoluteDirectory", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Directory: "coder/coder",
}, 0)
assert.Eventually(t, func() bool {
return client.GetStartup().Version != ""
}, testutil.WaitShort, testutil.IntervalFast)
startup := testutil.RequireRecvCtx(ctx, t, client.GetStartup())
homeDir, err := os.UserHomeDir()
require.NoError(t, err)
require.Equal(t, filepath.Join(homeDir, "coder/coder"), client.GetStartup().ExpandedDirectory)
require.Equal(t, filepath.Join(homeDir, "coder/coder"), startup.GetExpandedDirectory())
})
t.Run("HomeEnvironmentVariable", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Directory: "$HOME",
}, 0)
assert.Eventually(t, func() bool {
return client.GetStartup().Version != ""
}, testutil.WaitShort, testutil.IntervalFast)
startup := testutil.RequireRecvCtx(ctx, t, client.GetStartup())
homeDir, err := os.UserHomeDir()
require.NoError(t, err)
require.Equal(t, homeDir, client.GetStartup().ExpandedDirectory)
require.Equal(t, homeDir, startup.GetExpandedDirectory())
})
}
@@ -1521,7 +1606,7 @@ func TestAgent_ReconnectingPTY(t *testing.T) {
require.NoError(t, tr1.ReadUntil(ctx, matchPrompt), "find prompt")
require.NoError(t, tr2.ReadUntil(ctx, matchPrompt), "find prompt")
data, err := json.Marshal(codersdk.ReconnectingPTYRequest{
data, err := json.Marshal(workspacesdk.ReconnectingPTYRequest{
Data: "echo test\r",
})
require.NoError(t, err)
@@ -1549,7 +1634,7 @@ func TestAgent_ReconnectingPTY(t *testing.T) {
require.NoError(t, tr3.ReadUntil(ctx, matchEchoOutput), "find echo output")
// Exit should cause the connection to close.
data, err = json.Marshal(codersdk.ReconnectingPTYRequest{
data, err = json.Marshal(workspacesdk.ReconnectingPTYRequest{
Data: "exit\r",
})
require.NoError(t, err)
@@ -1635,9 +1720,10 @@ func TestAgent_Dial(t *testing.T) {
go func() {
defer close(done)
c, err := l.Accept()
assert.NoError(t, err, "accept connection")
defer c.Close()
testAccept(ctx, t, c)
if assert.NoError(t, err, "accept connection") {
defer c.Close()
testAccept(ctx, t, c)
}
}()
//nolint:dogsled
@@ -1662,11 +1748,13 @@ func TestAgent_UpdatedDERP(t *testing.T) {
require.NotNil(t, originalDerpMap)
coordinator := tailnet.NewCoordinator(logger)
defer func() {
// use t.Cleanup so the coordinator closing doesn't deadlock with in-memory
// coordination
t.Cleanup(func() {
_ = coordinator.Close()
}()
})
agentID := uuid.New()
statsCh := make(chan *agentsdk.Stats, 50)
statsCh := make(chan *proto.Stats, 50)
fs := afero.NewMemMapFs()
client := agenttest.NewClient(t,
logger.Named("agent"),
@@ -1679,49 +1767,57 @@ func TestAgent_UpdatedDERP(t *testing.T) {
statsCh,
coordinator,
)
closer := agent.New(agent.Options{
t.Cleanup(func() {
t.Log("closing client")
client.Close()
})
uut := agent.New(agent.Options{
Client: client,
Filesystem: fs,
Logger: logger.Named("agent"),
ReconnectingPTYTimeout: time.Minute,
})
defer func() {
_ = closer.Close()
}()
t.Cleanup(func() {
t.Log("closing agent")
_ = uut.Close()
})
// Setup a client connection.
newClientConn := func(derpMap *tailcfg.DERPMap) *codersdk.WorkspaceAgentConn {
newClientConn := func(derpMap *tailcfg.DERPMap, name string) *workspacesdk.AgentConn {
conn, err := tailnet.NewConn(&tailnet.Options{
Addresses: []netip.Prefix{netip.PrefixFrom(tailnet.IP(), 128)},
DERPMap: derpMap,
Logger: logger.Named("client"),
Logger: logger.Named(name),
})
require.NoError(t, err)
clientConn, serverConn := net.Pipe()
serveClientDone := make(chan struct{})
t.Cleanup(func() {
_ = clientConn.Close()
_ = serverConn.Close()
t.Logf("closing conn %s", name)
_ = conn.Close()
<-serveClientDone
})
go func() {
defer close(serveClientDone)
err := coordinator.ServeClient(serverConn, uuid.New(), agentID)
assert.NoError(t, err)
}()
sendNode, _ := tailnet.ServeCoordinator(clientConn, func(nodes []*tailnet.Node) error {
return conn.UpdateNodes(nodes, false)
testCtx, testCtxCancel := context.WithCancel(context.Background())
t.Cleanup(testCtxCancel)
clientID := uuid.New()
coordination := tailnet.NewInMemoryCoordination(
testCtx, logger,
clientID, agentID,
coordinator, conn)
t.Cleanup(func() {
t.Logf("closing coordination %s", name)
err := coordination.Close()
if err != nil {
t.Logf("error closing in-memory coordination: %s", err.Error())
}
t.Logf("closed coordination %s", name)
})
conn.SetNodeCallback(sendNode)
// Force DERP.
conn.SetBlockEndpoints(true)
sdkConn := codersdk.NewWorkspaceAgentConn(conn, codersdk.WorkspaceAgentConnOptions{
sdkConn := workspacesdk.NewAgentConn(conn, workspacesdk.AgentConnOptions{
AgentID: agentID,
CloseFunc: func() error { return codersdk.ErrSkipClose },
CloseFunc: func() error { return workspacesdk.ErrSkipClose },
})
t.Cleanup(func() {
t.Logf("closing sdkConn %s", name)
_ = sdkConn.Close()
})
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitLong)
@@ -1732,7 +1828,7 @@ func TestAgent_UpdatedDERP(t *testing.T) {
return sdkConn
}
conn1 := newClientConn(originalDerpMap)
conn1 := newClientConn(originalDerpMap, "client1")
// Change the DERP map.
newDerpMap, _ := tailnettest.RunDERPAndSTUN(t)
@@ -1747,31 +1843,36 @@ func TestAgent_UpdatedDERP(t *testing.T) {
}
// Push a new DERP map to the agent.
err := client.PushDERPMapUpdate(agentsdk.DERPMapUpdate{
DERPMap: newDerpMap,
})
err := client.PushDERPMapUpdate(newDerpMap)
require.NoError(t, err)
t.Logf("pushed DERPMap update to agent")
require.Eventually(t, func() bool {
conn := closer.TailnetConn()
conn := uut.TailnetConn()
if conn == nil {
return false
}
regionIDs := conn.DERPMap().RegionIDs()
return len(regionIDs) == 1 && regionIDs[0] == 2 && conn.Node().PreferredDERP == 2
preferredDERP := conn.Node().PreferredDERP
t.Logf("agent Conn DERPMap with regionIDs %v, PreferredDERP %d", regionIDs, preferredDERP)
return len(regionIDs) == 1 && regionIDs[0] == 2 && preferredDERP == 2
}, testutil.WaitLong, testutil.IntervalFast)
t.Logf("agent got the new DERPMap")
// Connect from a second client and make sure it uses the new DERP map.
conn2 := newClientConn(newDerpMap)
conn2 := newClientConn(newDerpMap, "client2")
require.Equal(t, []int{2}, conn2.DERPMap().RegionIDs())
t.Log("conn2 got the new DERPMap")
// If the first client gets a DERP map update, it should be able to
// reconnect just fine.
conn1.SetDERPMap(newDerpMap)
require.Equal(t, []int{2}, conn1.DERPMap().RegionIDs())
t.Log("set the new DERPMap on conn1")
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitLong)
defer cancel()
require.True(t, conn1.AwaitReachable(ctx))
t.Log("conn1 reached agent with new DERP")
}
func TestAgent_Speedtest(t *testing.T) {
@@ -1802,7 +1903,7 @@ func TestAgent_Reconnect(t *testing.T) {
defer coordinator.Close()
agentID := uuid.New()
statsCh := make(chan *agentsdk.Stats, 50)
statsCh := make(chan *proto.Stats, 50)
derpMap, _ := tailnettest.RunDERPAndSTUN(t)
client := agenttest.NewClient(t,
logger,
@@ -1813,6 +1914,7 @@ func TestAgent_Reconnect(t *testing.T) {
statsCh,
coordinator,
)
defer client.Close()
initialized := atomic.Int32{}
closer := agent.New(agent.Options{
ExchangeToken: func(ctx context.Context) (string, error) {
@@ -1846,9 +1948,10 @@ func TestAgent_WriteVSCodeConfigs(t *testing.T) {
GitAuthConfigs: 1,
DERPMap: &tailcfg.DERPMap{},
},
make(chan *agentsdk.Stats, 50),
make(chan *proto.Stats, 50),
coordinator,
)
defer client.Close()
filesystem := afero.NewMemMapFs()
closer := agent.New(agent.Options{
ExchangeToken: func(ctx context.Context) (string, error) {
@@ -1872,11 +1975,21 @@ func TestAgent_WriteVSCodeConfigs(t *testing.T) {
func TestAgent_DebugServer(t *testing.T) {
t.Parallel()
logDir := t.TempDir()
logPath := filepath.Join(logDir, "coder-agent.log")
randLogStr, err := cryptorand.String(32)
require.NoError(t, err)
require.NoError(t, os.WriteFile(logPath, []byte(randLogStr), 0o600))
derpMap, _ := tailnettest.RunDERPAndSTUN(t)
//nolint:dogsled
conn, _, _, _, agnt := setupAgent(t, agentsdk.Manifest{
DERPMap: derpMap,
}, 0)
}, 0, func(c *agenttest.Client, o *agent.Options) {
o.ExchangeToken = func(context.Context) (string, error) {
return "token", nil
}
o.LogDir = logDir
})
awaitReachableCtx := testutil.Context(t, testutil.WaitLong)
ok := conn.AwaitReachable(awaitReachableCtx)
@@ -1957,6 +2070,114 @@ func TestAgent_DebugServer(t *testing.T) {
require.Contains(t, string(resBody), `invalid state "blah", must be a boolean`)
})
})
t.Run("Manifest", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitLong)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL+"/debug/manifest", nil)
require.NoError(t, err)
res, err := srv.Client().Do(req)
require.NoError(t, err)
defer res.Body.Close()
require.Equal(t, http.StatusOK, res.StatusCode)
var v agentsdk.Manifest
require.NoError(t, json.NewDecoder(res.Body).Decode(&v))
require.NotNil(t, v)
})
t.Run("Logs", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitLong)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL+"/debug/logs", nil)
require.NoError(t, err)
res, err := srv.Client().Do(req)
require.NoError(t, err)
require.Equal(t, http.StatusOK, res.StatusCode)
defer res.Body.Close()
resBody, err := io.ReadAll(res.Body)
require.NoError(t, err)
require.NotEmpty(t, string(resBody))
require.Contains(t, string(resBody), randLogStr)
})
}
func TestAgent_ScriptLogging(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("bash scripts only")
}
t.Parallel()
ctx := testutil.Context(t, testutil.WaitMedium)
derpMap, _ := tailnettest.RunDERPAndSTUN(t)
logsCh := make(chan *proto.BatchCreateLogsRequest, 100)
lsStart := uuid.UUID{0x11}
lsStop := uuid.UUID{0x22}
//nolint:dogsled
_, _, _, _, agnt := setupAgent(
t,
agentsdk.Manifest{
DERPMap: derpMap,
Scripts: []codersdk.WorkspaceAgentScript{
{
LogSourceID: lsStart,
RunOnStart: true,
Script: `#!/bin/sh
i=0
while [ $i -ne 5 ]
do
i=$(($i+1))
echo "start $i"
done
`,
},
{
LogSourceID: lsStop,
RunOnStop: true,
Script: `#!/bin/sh
i=0
while [ $i -ne 3000 ]
do
i=$(($i+1))
echo "stop $i"
done
`, // send a lot of stop logs to make sure we don't truncate shutdown logs before closing the API conn
},
},
},
0,
func(cl *agenttest.Client, _ *agent.Options) {
cl.SetLogsChannel(logsCh)
},
)
n := 1
for n <= 5 {
logs := testutil.RequireRecvCtx(ctx, t, logsCh)
require.NotNil(t, logs)
for _, l := range logs.GetLogs() {
require.Equal(t, fmt.Sprintf("start %d", n), l.GetOutput())
n++
}
}
err := agnt.Close()
require.NoError(t, err)
n = 1
for n <= 3000 {
logs := testutil.RequireRecvCtx(ctx, t, logsCh)
require.NotNil(t, logs)
for _, l := range logs.GetLogs() {
require.Equal(t, fmt.Sprintf("stop %d", n), l.GetOutput())
n++
}
t.Logf("got %d stop logs", n-1)
}
}
// setupAgentSSHClient creates an agent, dials it, and sets up an ssh.Client for it
@@ -1974,15 +2195,17 @@ func setupSSHSession(
manifest agentsdk.Manifest,
serviceBanner codersdk.ServiceBannerConfig,
prepareFS func(fs afero.Fs),
opts ...func(*agenttest.Client, *agent.Options),
) *ssh.Session {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitLong)
defer cancel()
//nolint:dogsled
conn, _, _, fs, _ := setupAgent(t, manifest, 0, func(c *agenttest.Client, _ *agent.Options) {
opts = append(opts, func(c *agenttest.Client, o *agent.Options) {
c.SetServiceBannerFunc(func() (codersdk.ServiceBannerConfig, error) {
return serviceBanner, nil
})
})
//nolint:dogsled
conn, _, _, fs, _ := setupAgent(t, manifest, 0, opts...)
if prepareFS != nil {
prepareFS(fs)
}
@@ -2000,41 +2223,56 @@ func setupSSHSession(
}
func setupAgent(t *testing.T, metadata agentsdk.Manifest, ptyTimeout time.Duration, opts ...func(*agenttest.Client, *agent.Options)) (
*codersdk.WorkspaceAgentConn,
*workspacesdk.AgentConn,
*agenttest.Client,
<-chan *agentsdk.Stats,
<-chan *proto.Stats,
afero.Fs,
agent.Agent,
) {
logger := slogtest.Make(t, nil).Leveled(slog.LevelDebug)
logger := slogtest.Make(t, &slogtest.Options{
// Agent can drop errors when shutting down, and some, like the
// fasthttplistener connection closed error, are unexported.
IgnoreErrors: true,
}).Leveled(slog.LevelDebug)
if metadata.DERPMap == nil {
metadata.DERPMap, _ = tailnettest.RunDERPAndSTUN(t)
}
if metadata.AgentID == uuid.Nil {
metadata.AgentID = uuid.New()
}
if metadata.AgentName == "" {
metadata.AgentName = "test-agent"
}
if metadata.WorkspaceName == "" {
metadata.WorkspaceName = "test-workspace"
}
if metadata.WorkspaceID == uuid.Nil {
metadata.WorkspaceID = uuid.New()
}
coordinator := tailnet.NewCoordinator(logger)
t.Cleanup(func() {
_ = coordinator.Close()
})
statsCh := make(chan *agentsdk.Stats, 50)
statsCh := make(chan *proto.Stats, 50)
fs := afero.NewMemMapFs()
c := agenttest.NewClient(t, logger.Named("agent"), metadata.AgentID, metadata, statsCh, coordinator)
c := agenttest.NewClient(t, logger.Named("agenttest"), metadata.AgentID, metadata, statsCh, coordinator)
t.Cleanup(c.Close)
options := agent.Options{
Client: c,
Filesystem: fs,
Logger: logger.Named("agent"),
ReconnectingPTYTimeout: ptyTimeout,
EnvironmentVariables: map[string]string{},
}
for _, opt := range opts {
opt(c, &options)
}
closer := agent.New(options)
agnt := agent.New(options)
t.Cleanup(func() {
_ = closer.Close()
_ = agnt.Close()
})
conn, err := tailnet.NewConn(&tailnet.Options{
Addresses: []netip.Prefix{netip.PrefixFrom(tailnet.IP(), 128)},
@@ -2042,23 +2280,23 @@ func setupAgent(t *testing.T, metadata agentsdk.Manifest, ptyTimeout time.Durati
Logger: logger.Named("client"),
})
require.NoError(t, err)
clientConn, serverConn := net.Pipe()
serveClientDone := make(chan struct{})
t.Cleanup(func() {
_ = clientConn.Close()
_ = serverConn.Close()
_ = conn.Close()
<-serveClientDone
})
go func() {
defer close(serveClientDone)
coordinator.ServeClient(serverConn, uuid.New(), metadata.AgentID)
}()
sendNode, _ := tailnet.ServeCoordinator(clientConn, func(nodes []*tailnet.Node) error {
return conn.UpdateNodes(nodes, false)
testCtx, testCtxCancel := context.WithCancel(context.Background())
t.Cleanup(testCtxCancel)
clientID := uuid.New()
coordination := tailnet.NewInMemoryCoordination(
testCtx, logger,
clientID, metadata.AgentID,
coordinator, conn)
t.Cleanup(func() {
err := coordination.Close()
if err != nil {
t.Logf("error closing in-mem coordination: %s", err.Error())
}
})
conn.SetNodeCallback(sendNode)
agentConn := codersdk.NewWorkspaceAgentConn(conn, codersdk.WorkspaceAgentConnOptions{
agentConn := workspacesdk.NewAgentConn(conn, workspacesdk.AgentConnOptions{
AgentID: metadata.AgentID,
})
t.Cleanup(func() {
@@ -2071,7 +2309,7 @@ func setupAgent(t *testing.T, metadata agentsdk.Manifest, ptyTimeout time.Durati
if !agentConn.AwaitReachable(ctx) {
t.Fatal("agent not reachable")
}
return agentConn, c, statsCh, fs, closer
return agentConn, c, statsCh, fs, agnt
}
var dialTestPayload = []byte("dean-was-here123")
@@ -2291,11 +2529,11 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
logger = slog.Make(sloghuman.Sink(io.Discard))
)
requireFileWrite(t, fs, "/proc/self/oom_score_adj", "-500")
// Create some processes.
for i := 0; i < 4; i++ {
// Create a prioritized process. This process should
// have it's oom_score_adj set to -500 and its nice
// score should be untouched.
// Create a prioritized process.
var proc agentproc.Process
if i == 0 {
proc = agentproctest.GenerateProcess(t, fs,
@@ -2313,8 +2551,8 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
},
)
syscaller.EXPECT().SetPriority(proc.PID, 10).Return(nil)
syscaller.EXPECT().GetPriority(proc.PID).Return(20, nil)
syscaller.EXPECT().SetPriority(proc.PID, 10).Return(nil)
}
syscaller.EXPECT().
Kill(proc.PID, syscall.Signal(0)).
@@ -2333,6 +2571,9 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
})
actualProcs := <-modProcs
require.Len(t, actualProcs, len(expectedProcs)-1)
for _, proc := range actualProcs {
requireFileEquals(t, fs, fmt.Sprintf("/proc/%d/oom_score_adj", proc.PID), "0")
}
})
t.Run("IgnoreCustomNice", func(t *testing.T) {
@@ -2351,8 +2592,11 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
logger = slog.Make(sloghuman.Sink(io.Discard))
)
err := afero.WriteFile(fs, "/proc/self/oom_score_adj", []byte("0"), 0o644)
require.NoError(t, err)
// Create some processes.
for i := 0; i < 2; i++ {
for i := 0; i < 3; i++ {
proc := agentproctest.GenerateProcess(t, fs)
syscaller.EXPECT().
Kill(proc.PID, syscall.Signal(0)).
@@ -2380,7 +2624,59 @@ func TestAgent_ManageProcessPriority(t *testing.T) {
})
actualProcs := <-modProcs
// We should ignore the process with a custom nice score.
require.Len(t, actualProcs, 1)
require.Len(t, actualProcs, 2)
for _, proc := range actualProcs {
_, ok := expectedProcs[proc.PID]
require.True(t, ok)
requireFileEquals(t, fs, fmt.Sprintf("/proc/%d/oom_score_adj", proc.PID), "998")
}
})
t.Run("CustomOOMScore", func(t *testing.T) {
t.Parallel()
if runtime.GOOS != "linux" {
t.Skip("Skipping non-linux environment")
}
var (
fs = afero.NewMemMapFs()
ticker = make(chan time.Time)
syscaller = agentproctest.NewMockSyscaller(gomock.NewController(t))
modProcs = make(chan []*agentproc.Process)
logger = slog.Make(sloghuman.Sink(io.Discard))
)
err := afero.WriteFile(fs, "/proc/self/oom_score_adj", []byte("0"), 0o644)
require.NoError(t, err)
// Create some processes.
for i := 0; i < 3; i++ {
proc := agentproctest.GenerateProcess(t, fs)
syscaller.EXPECT().
Kill(proc.PID, syscall.Signal(0)).
Return(nil)
syscaller.EXPECT().GetPriority(proc.PID).Return(20, nil)
syscaller.EXPECT().SetPriority(proc.PID, 10).Return(nil)
}
_, _, _, _, _ = setupAgent(t, agentsdk.Manifest{}, 0, func(c *agenttest.Client, o *agent.Options) {
o.Syscaller = syscaller
o.ModifiedProcesses = modProcs
o.EnvironmentVariables = map[string]string{
agent.EnvProcPrioMgmt: "1",
agent.EnvProcOOMScore: "-567",
}
o.Filesystem = fs
o.Logger = logger
o.ProcessManagementTick = ticker
})
actualProcs := <-modProcs
// We should ignore the process with a custom nice score.
require.Len(t, actualProcs, 3)
for _, proc := range actualProcs {
requireFileEquals(t, fs, fmt.Sprintf("/proc/%d/oom_score_adj", proc.PID), "-567")
}
})
t.Run("DisabledByDefault", func(t *testing.T) {
@@ -2472,20 +2768,6 @@ func (s *syncWriter) Write(p []byte) (int, error) {
return s.w.Write(p)
}
// pickRandomPort picks a random port number for the ephemeral range. We do this entirely randomly
// instead of opening a listener and closing it to find a port that is likely to be free, since
// sometimes the OS reallocates the port very quickly.
func pickRandomPort() uint16 {
const (
// Overlap of windows, linux in https://en.wikipedia.org/wiki/Ephemeral_port
min = 49152
max = 60999
)
n := max - min
x := rand.Intn(n) //nolint: gosec
return uint16(min + x)
}
// echoOnce accepts a single connection, reads 4 bytes and echos them back
func echoOnce(t *testing.T, ll net.Listener) {
t.Helper()
@@ -2515,3 +2797,17 @@ func requireEcho(t *testing.T, conn net.Conn) {
require.NoError(t, err)
require.Equal(t, "test", string(b))
}
func requireFileWrite(t testing.TB, fs afero.Fs, fp, data string) {
t.Helper()
err := afero.WriteFile(fs, fp, []byte(data), 0o600)
require.NoError(t, err)
}
func requireFileEquals(t testing.TB, fs afero.Fs, fp, expect string) {
t.Helper()
actual, err := afero.ReadFile(fs, fp)
require.NoError(t, err)
require.Equal(t, expect, string(actual))
}
+8 -2
View File
@@ -2,6 +2,7 @@ package agentproctest
import (
"fmt"
"strconv"
"testing"
"github.com/spf13/afero"
@@ -29,8 +30,9 @@ func GenerateProcess(t *testing.T, fs afero.Fs, muts ...func(*agentproc.Process)
cmdline := fmt.Sprintf("%s\x00%s\x00%s", arg1, arg2, arg3)
process := agentproc.Process{
CmdLine: cmdline,
PID: int32(pid),
CmdLine: cmdline,
PID: int32(pid),
OOMScoreAdj: 0,
}
for _, mut := range muts {
@@ -45,5 +47,9 @@ func GenerateProcess(t *testing.T, fs afero.Fs, muts ...func(*agentproc.Process)
err = afero.WriteFile(fs, fmt.Sprintf("%s/cmdline", process.Dir), []byte(process.CmdLine), 0o444)
require.NoError(t, err)
score := strconv.Itoa(process.OOMScoreAdj)
err = afero.WriteFile(fs, fmt.Sprintf("%s/oom_score_adj", process.Dir), []byte(score), 0o444)
require.NoError(t, err)
return process
}
@@ -1,5 +1,10 @@
// Code generated by MockGen. DO NOT EDIT.
// Source: github.com/coder/coder/v2/agent/agentproc (interfaces: Syscaller)
//
// Generated by this command:
//
// mockgen -destination ./syscallermock.go -package agentproctest github.com/coder/coder/v2/agent/agentproc Syscaller
//
// Package agentproctest is a generated GoMock package.
package agentproctest
@@ -8,7 +13,7 @@ import (
reflect "reflect"
syscall "syscall"
gomock "github.com/golang/mock/gomock"
gomock "go.uber.org/mock/gomock"
)
// MockSyscaller is a mock of Syscaller interface.
@@ -44,7 +49,7 @@ func (m *MockSyscaller) GetPriority(arg0 int32) (int, error) {
}
// GetPriority indicates an expected call of GetPriority.
func (mr *MockSyscallerMockRecorder) GetPriority(arg0 interface{}) *gomock.Call {
func (mr *MockSyscallerMockRecorder) GetPriority(arg0 any) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetPriority", reflect.TypeOf((*MockSyscaller)(nil).GetPriority), arg0)
}
@@ -58,7 +63,7 @@ func (m *MockSyscaller) Kill(arg0 int32, arg1 syscall.Signal) error {
}
// Kill indicates an expected call of Kill.
func (mr *MockSyscallerMockRecorder) Kill(arg0, arg1 interface{}) *gomock.Call {
func (mr *MockSyscallerMockRecorder) Kill(arg0, arg1 any) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Kill", reflect.TypeOf((*MockSyscaller)(nil).Kill), arg0, arg1)
}
@@ -72,7 +77,7 @@ func (m *MockSyscaller) SetPriority(arg0 int32, arg1 int) error {
}
// SetPriority indicates an expected call of SetPriority.
func (mr *MockSyscallerMockRecorder) SetPriority(arg0, arg1 interface{}) *gomock.Call {
func (mr *MockSyscallerMockRecorder) SetPriority(arg0, arg1 any) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SetPriority", reflect.TypeOf((*MockSyscaller)(nil).SetPriority), arg0, arg1)
}
+1 -1
View File
@@ -5,9 +5,9 @@ import (
"syscall"
"testing"
"github.com/golang/mock/gomock"
"github.com/spf13/afero"
"github.com/stretchr/testify/require"
"go.uber.org/mock/gomock"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/agent/agentproc"
+20 -3
View File
@@ -5,6 +5,7 @@ package agentproc
import (
"errors"
"os"
"path/filepath"
"strconv"
"strings"
@@ -50,10 +51,26 @@ func List(fs afero.Fs, syscaller Syscaller) ([]*Process, error) {
}
return nil, xerrors.Errorf("read cmdline: %w", err)
}
oomScore, err := afero.ReadFile(fs, filepath.Join(defaultProcDir, entry, "oom_score_adj"))
if err != nil {
if xerrors.Is(err, os.ErrPermission) {
continue
}
return nil, xerrors.Errorf("read oom_score_adj: %w", err)
}
oom, err := strconv.Atoi(strings.TrimSpace(string(oomScore)))
if err != nil {
return nil, xerrors.Errorf("convert oom score: %w", err)
}
processes = append(processes, &Process{
PID: int32(pid),
CmdLine: string(cmdline),
Dir: filepath.Join(defaultProcDir, entry),
PID: int32(pid),
CmdLine: string(cmdline),
Dir: filepath.Join(defaultProcDir, entry),
OOMScoreAdj: oom,
})
}
+4 -3
View File
@@ -14,7 +14,8 @@ type Syscaller interface {
const defaultProcDir = "/proc"
type Process struct {
Dir string
CmdLine string
PID int32
Dir string
CmdLine string
PID int32
OOMScoreAdj int
}
+66 -11
View File
@@ -13,6 +13,7 @@ import (
"sync/atomic"
"time"
"github.com/google/uuid"
"github.com/prometheus/client_golang/prometheus"
"github.com/robfig/cron/v3"
"github.com/spf13/afero"
@@ -41,13 +42,19 @@ var (
parser = cron.NewParser(cron.Second | cron.Minute | cron.Hour | cron.Dom | cron.Month | cron.DowOptional)
)
type ScriptLogger interface {
Send(ctx context.Context, log ...agentsdk.Log) error
Flush(context.Context) error
}
// Options are a set of options for the runner.
type Options struct {
LogDir string
Logger slog.Logger
SSHServer *agentssh.Server
Filesystem afero.Fs
PatchLogs func(ctx context.Context, req agentsdk.PatchLogs) error
DataDirBase string
LogDir string
Logger slog.Logger
SSHServer *agentssh.Server
Filesystem afero.Fs
GetScriptLogger func(logSourceID uuid.UUID) ScriptLogger
}
// New creates a runner for the provided scripts.
@@ -59,6 +66,7 @@ func New(opts Options) *Runner {
cronCtxCancel: cronCtxCancel,
cron: cron.New(cron.WithParser(parser)),
closed: make(chan struct{}),
dataDir: filepath.Join(opts.DataDirBase, "coder-script-data"),
scriptsExecuted: prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: "agent",
Subsystem: "scripts",
@@ -78,6 +86,7 @@ type Runner struct {
cron *cron.Cron
initialized atomic.Bool
scripts []codersdk.WorkspaceAgentScript
dataDir string
// scriptsExecuted includes all scripts executed by the workspace agent. Agents
// execute startup scripts, and scripts on a cron schedule. Both will increment
@@ -85,6 +94,17 @@ type Runner struct {
scriptsExecuted *prometheus.CounterVec
}
// DataDir returns the directory where scripts data is stored.
func (r *Runner) DataDir() string {
return r.dataDir
}
// ScriptBinDir returns the directory where scripts can store executable
// binaries.
func (r *Runner) ScriptBinDir() string {
return filepath.Join(r.dataDir, "bin")
}
func (r *Runner) RegisterMetrics(reg prometheus.Registerer) {
if reg == nil {
// If no registry, do nothing.
@@ -104,6 +124,11 @@ func (r *Runner) Init(scripts []codersdk.WorkspaceAgentScript) error {
r.scripts = scripts
r.Logger.Info(r.cronCtx, "initializing agent scripts", slog.F("script_count", len(scripts)), slog.F("log_dir", r.LogDir))
err := r.Filesystem.MkdirAll(r.ScriptBinDir(), 0o700)
if err != nil {
return xerrors.Errorf("create script bin dir: %w", err)
}
for _, script := range scripts {
if script.Cron == "" {
continue
@@ -129,7 +154,18 @@ func (r *Runner) StartCron() {
// has exited by the time the `cron.Stop()` context returns, so we need to
// track it manually.
err := r.trackCommandGoroutine(func() {
r.cron.Run()
// Since this is run async, in quick unit tests, it is possible the
// Close() function gets called before we even start the cron.
// In these cases, the Run() will never end.
// So if we are closed, we just return, and skip the Run() entirely.
select {
case <-r.cronCtx.Done():
// The cronCtx is canceled before cron.Close() happens. So if the ctx is
// canceled, then Close() will be called, or it is about to be called.
// So do nothing!
default:
r.cron.Run()
}
})
if err != nil {
r.Logger.Warn(context.Background(), "start cron failed", slog.Error(err))
@@ -197,7 +233,18 @@ func (r *Runner) run(ctx context.Context, script codersdk.WorkspaceAgentScript)
if !filepath.IsAbs(logPath) {
logPath = filepath.Join(r.LogDir, logPath)
}
logger := r.Logger.With(slog.F("log_path", logPath))
scriptDataDir := filepath.Join(r.DataDir(), script.LogSourceID.String())
err := r.Filesystem.MkdirAll(scriptDataDir, 0o700)
if err != nil {
return xerrors.Errorf("%s script: create script temp dir: %w", scriptDataDir, err)
}
logger := r.Logger.With(
slog.F("log_source_id", script.LogSourceID),
slog.F("log_path", logPath),
slog.F("script_data_dir", scriptDataDir),
)
logger.Info(ctx, "running agent script", slog.F("script", script.Script))
fileWriter, err := r.Filesystem.OpenFile(logPath, os.O_CREATE|os.O_RDWR, 0o600)
@@ -227,20 +274,27 @@ func (r *Runner) run(ctx context.Context, script codersdk.WorkspaceAgentScript)
cmd.WaitDelay = 10 * time.Second
cmd.Cancel = cmdCancel(cmd)
send, flushAndClose := agentsdk.LogsSender(script.LogSourceID, r.PatchLogs, logger)
// Expose env vars that can be used in the script for storing data
// and binaries. In the future, we may want to expose more env vars
// for the script to use, like CODER_SCRIPT_DATA_DIR for persistent
// storage.
cmd.Env = append(cmd.Env, "CODER_SCRIPT_DATA_DIR="+scriptDataDir)
cmd.Env = append(cmd.Env, "CODER_SCRIPT_BIN_DIR="+r.ScriptBinDir())
scriptLogger := r.GetScriptLogger(script.LogSourceID)
// If ctx is canceled here (or in a writer below), we may be
// discarding logs, but that's okay because we're shutting down
// anyway. We could consider creating a new context here if we
// want better control over flush during shutdown.
defer func() {
if err := flushAndClose(ctx); err != nil {
if err := scriptLogger.Flush(ctx); err != nil {
logger.Warn(ctx, "flush startup logs failed", slog.Error(err))
}
}()
infoW := agentsdk.LogsWriter(ctx, send, script.LogSourceID, codersdk.LogLevelInfo)
infoW := agentsdk.LogsWriter(ctx, scriptLogger.Send, script.LogSourceID, codersdk.LogLevelInfo)
defer infoW.Close()
errW := agentsdk.LogsWriter(ctx, send, script.LogSourceID, codersdk.LogLevelError)
errW := agentsdk.LogsWriter(ctx, scriptLogger.Send, script.LogSourceID, codersdk.LogLevelError)
defer errW.Close()
cmd.Stdout = io.MultiWriter(fileWriter, infoW)
cmd.Stderr = io.MultiWriter(fileWriter, errW)
@@ -315,6 +369,7 @@ func (r *Runner) Close() error {
return nil
}
close(r.closed)
// Must cancel the cron ctx BEFORE stopping the cron.
r.cronCtxCancel()
<-r.cron.Stop().Done()
r.cmdCloseWait.Wait()
+123 -22
View File
@@ -2,13 +2,16 @@ package agentscripts_test
import (
"context"
"path/filepath"
"runtime"
"testing"
"time"
"github.com/google/uuid"
"github.com/prometheus/client_golang/prometheus"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/atomic"
"go.uber.org/goleak"
"cdr.dev/slog/sloggers/slogtest"
@@ -16,6 +19,7 @@ import (
"github.com/coder/coder/v2/agent/agentssh"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/testutil"
)
func TestMain(m *testing.M) {
@@ -24,21 +28,75 @@ func TestMain(m *testing.M) {
func TestExecuteBasic(t *testing.T) {
t.Parallel()
logs := make(chan agentsdk.PatchLogs, 1)
runner := setup(t, func(ctx context.Context, req agentsdk.PatchLogs) error {
logs <- req
return nil
ctx := testutil.Context(t, testutil.WaitShort)
fLogger := newFakeScriptLogger()
runner := setup(t, func(uuid2 uuid.UUID) agentscripts.ScriptLogger {
return fLogger
})
defer runner.Close()
err := runner.Init([]codersdk.WorkspaceAgentScript{{
Script: "echo hello",
LogSourceID: uuid.New(),
Script: "echo hello",
}})
require.NoError(t, err)
require.NoError(t, runner.Execute(context.Background(), func(script codersdk.WorkspaceAgentScript) bool {
return true
}))
log := <-logs
require.Equal(t, "hello", log.Logs[0].Output)
log := testutil.RequireRecvCtx(ctx, t, fLogger.logs)
require.Equal(t, "hello", log.Output)
}
func TestEnv(t *testing.T) {
t.Parallel()
fLogger := newFakeScriptLogger()
runner := setup(t, func(uuid2 uuid.UUID) agentscripts.ScriptLogger {
return fLogger
})
defer runner.Close()
id := uuid.New()
script := "echo $CODER_SCRIPT_DATA_DIR\necho $CODER_SCRIPT_BIN_DIR\n"
if runtime.GOOS == "windows" {
script = `
cmd.exe /c echo %CODER_SCRIPT_DATA_DIR%
cmd.exe /c echo %CODER_SCRIPT_BIN_DIR%
`
}
err := runner.Init([]codersdk.WorkspaceAgentScript{{
LogSourceID: id,
Script: script,
}})
require.NoError(t, err)
ctx := testutil.Context(t, testutil.WaitLong)
done := testutil.Go(t, func() {
err := runner.Execute(ctx, func(script codersdk.WorkspaceAgentScript) bool {
return true
})
assert.NoError(t, err)
})
defer func() {
select {
case <-ctx.Done():
case <-done:
}
}()
var log []agentsdk.Log
for {
select {
case <-ctx.Done():
require.Fail(t, "timed out waiting for logs")
case l := <-fLogger.logs:
t.Logf("log: %s", l.Output)
log = append(log, l)
}
if len(log) >= 2 {
break
}
}
require.Contains(t, log[0].Output, filepath.Join(runner.DataDir(), id.String()))
require.Contains(t, log[1].Output, runner.ScriptBinDir())
}
func TestTimeout(t *testing.T) {
@@ -46,35 +104,78 @@ func TestTimeout(t *testing.T) {
runner := setup(t, nil)
defer runner.Close()
err := runner.Init([]codersdk.WorkspaceAgentScript{{
Script: "sleep infinity",
Timeout: time.Millisecond,
LogSourceID: uuid.New(),
Script: "sleep infinity",
Timeout: time.Millisecond,
}})
require.NoError(t, err)
require.ErrorIs(t, runner.Execute(context.Background(), nil), agentscripts.ErrTimeout)
}
func setup(t *testing.T, patchLogs func(ctx context.Context, req agentsdk.PatchLogs) error) *agentscripts.Runner {
// TestCronClose exists because cron.Run() can happen after cron.Close().
// If this happens, there used to be a deadlock.
func TestCronClose(t *testing.T) {
t.Parallel()
runner := agentscripts.New(agentscripts.Options{})
runner.StartCron()
require.NoError(t, runner.Close(), "close runner")
}
func setup(t *testing.T, getScriptLogger func(logSourceID uuid.UUID) agentscripts.ScriptLogger) *agentscripts.Runner {
t.Helper()
if patchLogs == nil {
if getScriptLogger == nil {
// noop
patchLogs = func(ctx context.Context, req agentsdk.PatchLogs) error {
return nil
getScriptLogger = func(uuid uuid.UUID) agentscripts.ScriptLogger {
return noopScriptLogger{}
}
}
fs := afero.NewMemMapFs()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(context.Background(), logger, prometheus.NewRegistry(), fs, 0, "")
s, err := agentssh.NewServer(context.Background(), logger, prometheus.NewRegistry(), fs, nil)
require.NoError(t, err)
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
t.Cleanup(func() {
_ = s.Close()
})
return agentscripts.New(agentscripts.Options{
LogDir: t.TempDir(),
Logger: logger,
SSHServer: s,
Filesystem: fs,
PatchLogs: patchLogs,
LogDir: t.TempDir(),
DataDirBase: t.TempDir(),
Logger: logger,
SSHServer: s,
Filesystem: fs,
GetScriptLogger: getScriptLogger,
})
}
type noopScriptLogger struct{}
func (noopScriptLogger) Send(context.Context, ...agentsdk.Log) error {
return nil
}
func (noopScriptLogger) Flush(context.Context) error {
return nil
}
type fakeScriptLogger struct {
logs chan agentsdk.Log
}
func (f *fakeScriptLogger) Send(ctx context.Context, logs ...agentsdk.Log) error {
for _, log := range logs {
select {
case <-ctx.Done():
return ctx.Err()
case f.logs <- log:
// OK!
}
}
return nil
}
func (*fakeScriptLogger) Flush(context.Context) error {
return nil
}
func newFakeScriptLogger() *fakeScriptLogger {
return &fakeScriptLogger{make(chan agentsdk.Log, 100)}
}
+74 -74
View File
@@ -32,7 +32,6 @@ import (
"github.com/coder/coder/v2/agent/usershell"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/pty"
)
@@ -55,6 +54,28 @@ const (
MagicProcessCmdlineJetBrains = "idea.vendor.name=JetBrains"
)
// Config sets configuration parameters for the agent SSH server.
type Config struct {
// MaxTimeout sets the absolute connection timeout, none if empty. If set to
// 3 seconds or more, keep alive will be used instead.
MaxTimeout time.Duration
// MOTDFile returns the path to the message of the day file. If set, the
// file will be displayed to the user upon login.
MOTDFile func() string
// ServiceBanner returns the configuration for the Coder service banner.
ServiceBanner func() *codersdk.ServiceBannerConfig
// UpdateEnv updates the environment variables for the command to be
// executed. It can be used to add, modify or replace environment variables.
UpdateEnv func(current []string) (updated []string, err error)
// WorkingDirectory sets the working directory for commands and defines
// where users will land when they connect via SSH. Default is the home
// directory of the user.
WorkingDirectory func() string
// X11SocketDir is the directory where X11 sockets are created. Default is
// /tmp/.X11-unix.
X11SocketDir string
}
type Server struct {
mu sync.RWMutex // Protects following.
fs afero.Fs
@@ -66,14 +87,10 @@ type Server struct {
// a lock on mu but protected by closing.
wg sync.WaitGroup
logger slog.Logger
srv *ssh.Server
x11SocketDir string
logger slog.Logger
srv *ssh.Server
Env map[string]string
AgentToken func() string
Manifest *atomic.Pointer[agentsdk.Manifest]
ServiceBanner *atomic.Pointer[codersdk.ServiceBannerConfig]
config *Config
connCountVSCode atomic.Int64
connCountJetBrains atomic.Int64
@@ -82,7 +99,7 @@ type Server struct {
metrics *sshServerMetrics
}
func NewServer(ctx context.Context, logger slog.Logger, prometheusRegistry *prometheus.Registry, fs afero.Fs, maxTimeout time.Duration, x11SocketDir string) (*Server, error) {
func NewServer(ctx context.Context, logger slog.Logger, prometheusRegistry *prometheus.Registry, fs afero.Fs, config *Config) (*Server, error) {
// Clients' should ignore the host key when connecting.
// The agent needs to authenticate with coderd to SSH,
// so SSH authentication doesn't improve security.
@@ -94,21 +111,43 @@ func NewServer(ctx context.Context, logger slog.Logger, prometheusRegistry *prom
if err != nil {
return nil, err
}
if x11SocketDir == "" {
x11SocketDir = filepath.Join(os.TempDir(), ".X11-unix")
if config == nil {
config = &Config{}
}
if config.X11SocketDir == "" {
config.X11SocketDir = filepath.Join(os.TempDir(), ".X11-unix")
}
if config.UpdateEnv == nil {
config.UpdateEnv = func(current []string) ([]string, error) { return current, nil }
}
if config.MOTDFile == nil {
config.MOTDFile = func() string { return "" }
}
if config.ServiceBanner == nil {
config.ServiceBanner = func() *codersdk.ServiceBannerConfig { return &codersdk.ServiceBannerConfig{} }
}
if config.WorkingDirectory == nil {
config.WorkingDirectory = func() string {
home, err := userHomeDir()
if err != nil {
return ""
}
return home
}
}
forwardHandler := &ssh.ForwardedTCPHandler{}
unixForwardHandler := &forwardedUnixHandler{log: logger}
unixForwardHandler := newForwardedUnixHandler(logger)
metrics := newSSHServerMetrics(prometheusRegistry)
s := &Server{
listeners: make(map[net.Listener]struct{}),
fs: fs,
conns: make(map[net.Conn]struct{}),
sessions: make(map[ssh.Session]struct{}),
logger: logger,
x11SocketDir: x11SocketDir,
listeners: make(map[net.Listener]struct{}),
fs: fs,
conns: make(map[net.Conn]struct{}),
sessions: make(map[ssh.Session]struct{}),
logger: logger,
config: config,
metrics: metrics,
}
@@ -172,14 +211,16 @@ func NewServer(ctx context.Context, logger slog.Logger, prometheusRegistry *prom
},
}
// The MaxTimeout functionality has been substituted with the introduction of the KeepAlive feature.
// In cases where very short timeouts are set, the SSH server will automatically switch to the connection timeout for both read and write operations.
if maxTimeout >= 3*time.Second {
// The MaxTimeout functionality has been substituted with the introduction
// of the KeepAlive feature. In cases where very short timeouts are set, the
// SSH server will automatically switch to the connection timeout for both
// read and write operations.
if config.MaxTimeout >= 3*time.Second {
srv.ClientAliveCountMax = 3
srv.ClientAliveInterval = maxTimeout / time.Duration(srv.ClientAliveCountMax)
srv.ClientAliveInterval = config.MaxTimeout / time.Duration(srv.ClientAliveCountMax)
srv.MaxTimeout = 0
} else {
srv.MaxTimeout = maxTimeout
srv.MaxTimeout = config.MaxTimeout
}
s.srv = srv
@@ -400,7 +441,7 @@ func (s *Server) startPTYSession(logger slog.Logger, session ptySession, magicTy
session.DisablePTYEmulation()
if isLoginShell(session.RawCommand()) {
serviceBanner := s.ServiceBanner.Load()
serviceBanner := s.config.ServiceBanner()
if serviceBanner != nil {
err := showServiceBanner(session, serviceBanner)
if err != nil {
@@ -411,15 +452,10 @@ func (s *Server) startPTYSession(logger slog.Logger, session ptySession, magicTy
}
if !isQuietLogin(s.fs, session.RawCommand()) {
manifest := s.Manifest.Load()
if manifest != nil {
err := showMOTD(s.fs, session, manifest.MOTDFile)
if err != nil {
logger.Error(ctx, "agent failed to show MOTD", slog.Error(err))
s.metrics.sessionErrors.WithLabelValues(magicTypeLabel, "yes", "motd").Add(1)
}
} else {
logger.Warn(ctx, "metadata lookup failed, unable to show MOTD")
err := showMOTD(s.fs, session, s.config.MOTDFile())
if err != nil {
logger.Error(ctx, "agent failed to show MOTD", slog.Error(err))
s.metrics.sessionErrors.WithLabelValues(magicTypeLabel, "yes", "motd").Add(1)
}
}
@@ -557,7 +593,7 @@ func (s *Server) sftpHandler(logger slog.Logger, session ssh.Session) {
defer server.Close()
err = server.Serve()
if errors.Is(err, io.EOF) {
if err == nil || errors.Is(err, io.EOF) {
// Unless we call `session.Exit(0)` here, the client won't
// receive `exit-status` because `(*sftp.Server).Close()`
// calls `Close()` on the underlying connection (session),
@@ -589,11 +625,6 @@ func (s *Server) CreateCommand(ctx context.Context, script string, env []string)
return nil, xerrors.Errorf("get user shell: %w", err)
}
manifest := s.Manifest.Load()
if manifest == nil {
return nil, xerrors.Errorf("no metadata was provided")
}
// OpenSSH executes all commands with the users current shell.
// We replicate that behavior for IDE support.
caller := "-c"
@@ -638,7 +669,7 @@ func (s *Server) CreateCommand(ctx context.Context, script string, env []string)
}
cmd := pty.CommandContext(ctx, name, args...)
cmd.Dir = manifest.Directory
cmd.Dir = s.config.WorkingDirectory()
// If the metadata directory doesn't exist, we run the command
// in the users home directory.
@@ -652,21 +683,7 @@ func (s *Server) CreateCommand(ctx context.Context, script string, env []string)
cmd.Dir = homedir
}
cmd.Env = append(os.Environ(), env...)
executablePath, err := os.Executable()
if err != nil {
return nil, xerrors.Errorf("getting os executable: %w", err)
}
// Set environment variables reliable detection of being inside a
// Coder workspace.
cmd.Env = append(cmd.Env, "CODER=true")
cmd.Env = append(cmd.Env, fmt.Sprintf("USER=%s", username))
// Git on Windows resolves with UNIX-style paths.
// If using backslashes, it's unable to find the executable.
unixExecutablePath := strings.ReplaceAll(executablePath, "\\", "/")
cmd.Env = append(cmd.Env, fmt.Sprintf(`GIT_SSH_COMMAND=%s gitssh --`, unixExecutablePath))
// Specific Coder subcommands require the agent token exposed!
cmd.Env = append(cmd.Env, fmt.Sprintf("CODER_AGENT_TOKEN=%s", s.AgentToken()))
// Set SSH connection environment variables (these are also set by OpenSSH
// and thus expected to be present by SSH clients). Since the agent does
@@ -677,26 +694,9 @@ func (s *Server) CreateCommand(ctx context.Context, script string, env []string)
cmd.Env = append(cmd.Env, fmt.Sprintf("SSH_CLIENT=%s %s %s", srcAddr, srcPort, dstPort))
cmd.Env = append(cmd.Env, fmt.Sprintf("SSH_CONNECTION=%s %s %s %s", srcAddr, srcPort, dstAddr, dstPort))
// This adds the ports dialog to code-server that enables
// proxying a port dynamically.
cmd.Env = append(cmd.Env, fmt.Sprintf("VSCODE_PROXY_URI=%s", manifest.VSCodePortProxyURI))
// Hide Coder message on code-server's "Getting Started" page
cmd.Env = append(cmd.Env, "CS_DISABLE_GETTING_STARTED_OVERRIDE=true")
// Load environment variables passed via the agent.
// These should override all variables we manually specify.
for envKey, value := range manifest.EnvironmentVariables {
// Expanding environment variables allows for customization
// of the $PATH, among other variables. Customers can prepend
// or append to the $PATH, so allowing expand is required!
cmd.Env = append(cmd.Env, fmt.Sprintf("%s=%s", envKey, os.ExpandEnv(value)))
}
// Agent-level environment variables should take over all!
// This is used for setting agent-specific variables like "CODER_AGENT_TOKEN".
for envKey, value := range s.Env {
cmd.Env = append(cmd.Env, fmt.Sprintf("%s=%s", envKey, value))
cmd.Env, err = s.config.UpdateEnv(cmd.Env)
if err != nil {
return nil, xerrors.Errorf("apply env: %w", err)
}
return cmd, nil
+1 -1
View File
@@ -37,7 +37,7 @@ func Test_sessionStart_orphan(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitMedium)
defer cancel()
logger := slogtest.Make(t, nil)
s, err := NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
+5 -25
View File
@@ -17,14 +17,12 @@ import (
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/atomic"
"go.uber.org/goleak"
"golang.org/x/crypto/ssh"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent/agentssh"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/coder/v2/testutil"
)
@@ -38,14 +36,10 @@ func TestNewServer_ServeClient(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
@@ -83,13 +77,11 @@ func TestNewServer_ExecuteShebang(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
t.Cleanup(func() {
_ = s.Close()
})
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
t.Run("Basic", func(t *testing.T) {
t.Parallel()
@@ -116,14 +108,10 @@ func TestNewServer_CloseActiveConnections(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, &slogtest.Options{IgnoreErrors: true})
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
@@ -171,14 +159,10 @@ func TestNewServer_Signal(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
@@ -240,14 +224,10 @@ func TestNewServer_Signal(t *testing.T) {
ctx := context.Background()
logger := slogtest.Make(t, nil)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), 0, "")
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), afero.NewMemMapFs(), nil)
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
+76 -35
View File
@@ -2,11 +2,14 @@ package agentssh
import (
"context"
"errors"
"fmt"
"io/fs"
"net"
"os"
"path/filepath"
"sync"
"syscall"
"github.com/gliderlabs/ssh"
gossh "golang.org/x/crypto/ssh"
@@ -33,22 +36,29 @@ type forwardedStreamLocalPayload struct {
type forwardedUnixHandler struct {
sync.Mutex
log slog.Logger
forwards map[string]net.Listener
forwards map[forwardKey]net.Listener
}
type forwardKey struct {
sessionID string
addr string
}
func newForwardedUnixHandler(log slog.Logger) *forwardedUnixHandler {
return &forwardedUnixHandler{
log: log,
forwards: make(map[forwardKey]net.Listener),
}
}
func (h *forwardedUnixHandler) HandleSSHRequest(ctx ssh.Context, _ *ssh.Server, req *gossh.Request) (bool, []byte) {
h.log.Debug(ctx, "handling SSH unix forward")
h.Lock()
if h.forwards == nil {
h.forwards = make(map[string]net.Listener)
}
h.Unlock()
conn, ok := ctx.Value(ssh.ContextKeyConn).(*gossh.ServerConn)
if !ok {
h.log.Warn(ctx, "SSH unix forward request from client with no gossh connection")
return false, nil
}
log := h.log.With(slog.F("remote_addr", conn.RemoteAddr()))
log := h.log.With(slog.F("session_id", ctx.SessionID()), slog.F("remote_addr", conn.RemoteAddr()))
switch req.Type {
case "streamlocal-forward@openssh.com":
@@ -62,14 +72,22 @@ func (h *forwardedUnixHandler) HandleSSHRequest(ctx ssh.Context, _ *ssh.Server,
addr := reqPayload.SocketPath
log = log.With(slog.F("socket_path", addr))
log.Debug(ctx, "request begin SSH unix forward")
key := forwardKey{
sessionID: ctx.SessionID(),
addr: addr,
}
h.Lock()
_, ok := h.forwards[addr]
_, ok := h.forwards[key]
h.Unlock()
if ok {
log.Warn(ctx, "SSH unix forward request for socket path that is already being forwarded (maybe to another client?)",
slog.F("socket_path", addr),
)
return false, nil
// In cases where `ExitOnForwardFailure=yes` is set, returning false
// here will cause the connection to be closed. To avoid this, and
// to match OpenSSH behavior, we silently ignore the second forward
// request.
log.Warn(ctx, "SSH unix forward request for socket path that is already being forwarded on this session, ignoring")
return true, nil
}
// Create socket parent dir if not exists.
@@ -83,12 +101,20 @@ func (h *forwardedUnixHandler) HandleSSHRequest(ctx ssh.Context, _ *ssh.Server,
return false, nil
}
ln, err := net.Listen("unix", addr)
// Remove existing socket if it exists. We do not use os.Remove() here
// so that directories are kept. Note that it's possible that we will
// overwrite a regular file here. Both of these behaviors match OpenSSH,
// however, which is why we unlink.
err = unlink(addr)
if err != nil && !errors.Is(err, fs.ErrNotExist) {
log.Warn(ctx, "remove existing socket for SSH unix forward request", slog.Error(err))
return false, nil
}
lc := &net.ListenConfig{}
ln, err := lc.Listen(ctx, "unix", addr)
if err != nil {
log.Warn(ctx, "listen on Unix socket for SSH unix forward request",
slog.F("socket_path", addr),
slog.Error(err),
)
log.Warn(ctx, "listen on Unix socket for SSH unix forward request", slog.Error(err))
return false, nil
}
log.Debug(ctx, "SSH unix forward listening on socket")
@@ -99,7 +125,7 @@ func (h *forwardedUnixHandler) HandleSSHRequest(ctx ssh.Context, _ *ssh.Server,
//
// This is also what the upstream TCP version of this code does.
h.Lock()
h.forwards[addr] = ln
h.forwards[key] = ln
h.Unlock()
log.Debug(ctx, "SSH unix forward added to cache")
@@ -115,9 +141,7 @@ func (h *forwardedUnixHandler) HandleSSHRequest(ctx ssh.Context, _ *ssh.Server,
c, err := ln.Accept()
if err != nil {
if !xerrors.Is(err, net.ErrClosed) {
log.Warn(ctx, "accept on local Unix socket for SSH unix forward request",
slog.Error(err),
)
log.Warn(ctx, "accept on local Unix socket for SSH unix forward request", slog.Error(err))
}
// closed below
log.Debug(ctx, "SSH unix forward listener closed")
@@ -131,10 +155,7 @@ func (h *forwardedUnixHandler) HandleSSHRequest(ctx ssh.Context, _ *ssh.Server,
go func() {
ch, reqs, err := conn.OpenChannel("forwarded-streamlocal@openssh.com", payload)
if err != nil {
h.log.Warn(ctx, "open SSH unix forward channel to client",
slog.F("socket_path", addr),
slog.Error(err),
)
h.log.Warn(ctx, "open SSH unix forward channel to client", slog.Error(err))
_ = c.Close()
return
}
@@ -144,12 +165,11 @@ func (h *forwardedUnixHandler) HandleSSHRequest(ctx ssh.Context, _ *ssh.Server,
}
h.Lock()
ln2, ok := h.forwards[addr]
if ok && ln2 == ln {
delete(h.forwards, addr)
if ln2, ok := h.forwards[key]; ok && ln2 == ln {
delete(h.forwards, key)
}
h.Unlock()
log.Debug(ctx, "SSH unix forward listener removed from cache", slog.F("path", addr))
log.Debug(ctx, "SSH unix forward listener removed from cache")
_ = ln.Close()
}()
@@ -162,13 +182,22 @@ func (h *forwardedUnixHandler) HandleSSHRequest(ctx ssh.Context, _ *ssh.Server,
h.log.Warn(ctx, "parse cancel-streamlocal-forward@openssh.com (SSH unix forward) request payload from client", slog.Error(err))
return false, nil
}
log.Debug(ctx, "request to cancel SSH unix forward", slog.F("path", reqPayload.SocketPath))
h.Lock()
ln, ok := h.forwards[reqPayload.SocketPath]
h.Unlock()
if ok {
_ = ln.Close()
log.Debug(ctx, "request to cancel SSH unix forward", slog.F("socket_path", reqPayload.SocketPath))
key := forwardKey{
sessionID: ctx.SessionID(),
addr: reqPayload.SocketPath,
}
h.Lock()
ln, ok := h.forwards[key]
delete(h.forwards, key)
h.Unlock()
if !ok {
log.Warn(ctx, "SSH unix forward not found in cache")
return true, nil
}
_ = ln.Close()
return true, nil
default:
@@ -209,3 +238,15 @@ func directStreamLocalHandler(_ *ssh.Server, _ *gossh.ServerConn, newChan gossh.
Bicopy(ctx, ch, dconn)
}
// unlink removes files and unlike os.Remove, directories are kept.
func unlink(path string) error {
// Ignore EINTR like os.Remove, see ignoringEINTR in os/file_posix.go
// for more details.
for {
err := syscall.Unlink(path)
if !errors.Is(err, syscall.EINTR) {
return err
}
}
}
+7
View File
@@ -1,6 +1,7 @@
package agentssh
import (
"context"
"strings"
"sync"
@@ -26,6 +27,7 @@ type localForwardChannelData struct {
type JetbrainsChannelWatcher struct {
gossh.NewChannel
jetbrainsCounter *atomic.Int64
logger slog.Logger
}
func NewJetbrainsChannelWatcher(ctx ssh.Context, logger slog.Logger, newChannel gossh.NewChannel, counter *atomic.Int64) gossh.NewChannel {
@@ -58,6 +60,7 @@ func NewJetbrainsChannelWatcher(ctx ssh.Context, logger slog.Logger, newChannel
return &JetbrainsChannelWatcher{
NewChannel: newChannel,
jetbrainsCounter: counter,
logger: logger.With(slog.F("destination_port", d.DestPort)),
}
}
@@ -67,11 +70,15 @@ func (w *JetbrainsChannelWatcher) Accept() (gossh.Channel, <-chan *gossh.Request
return c, r, err
}
w.jetbrainsCounter.Add(1)
// nolint: gocritic // JetBrains is a proper noun and should be capitalized
w.logger.Debug(context.Background(), "JetBrains watcher accepted channel")
return &ChannelOnClose{
Channel: c,
done: func() {
w.jetbrainsCounter.Add(-1)
// nolint: gocritic // JetBrains is a proper noun and should be capitalized
w.logger.Debug(context.Background(), "JetBrains watcher channel closed")
},
}, r, err
}
+23 -9
View File
@@ -3,6 +3,7 @@
package agentssh
import (
"errors"
"fmt"
"os"
@@ -11,24 +12,37 @@ import (
)
func getListeningPortProcessCmdline(port uint32) (string, error) {
tabs, err := netstat.TCPSocks(func(s *netstat.SockTabEntry) bool {
acceptFn := func(s *netstat.SockTabEntry) bool {
return s.LocalAddr != nil && uint32(s.LocalAddr.Port) == port
})
if err != nil {
return "", xerrors.Errorf("inspect port %d: %w", port, err)
}
if len(tabs) == 0 {
return "", nil
tabs4, err4 := netstat.TCPSocks(acceptFn)
tabs6, err6 := netstat.TCP6Socks(acceptFn)
// In the common case, we want to check ipv4 listening addresses. If this
// fails, we should return an error. We also need to check ipv6. The
// assumption is, if we have an err4, and 0 ipv6 addresses listed, then we are
// interested in the err4 (and vice versa). So return both errors (at least 1
// is non-nil) if the other list is empty.
if (err4 != nil && len(tabs6) == 0) || (err6 != nil && len(tabs4) == 0) {
return "", xerrors.Errorf("inspect port %d: %w", port, errors.Join(err4, err6))
}
// Defensive check.
if tabs[0].Process == nil {
var proc *netstat.Process
if len(tabs4) > 0 {
proc = tabs4[0].Process
} else if len(tabs6) > 0 {
proc = tabs6[0].Process
}
if proc == nil {
// Either nothing is listening on this port or we were unable to read the
// process details (permission issues reading /proc/$pid/* potentially).
// Or, perhaps /proc/net/tcp{,6} is not listing the port for some reason.
return "", nil
}
// The process name provided by go-netstat does not include the full command
// line so grab that instead.
pid := tabs[0].Process.Pid
pid := proc.Pid
data, err := os.ReadFile(fmt.Sprintf("/proc/%d/cmdline", pid))
if err != nil {
return "", xerrors.Errorf("read /proc/%d/cmdline: %w", pid, err)
+197 -5
View File
@@ -6,6 +6,7 @@ import (
"encoding/hex"
"errors"
"fmt"
"io"
"net"
"os"
"path/filepath"
@@ -31,9 +32,9 @@ func (s *Server) x11Callback(ctx ssh.Context, x11 ssh.X11) bool {
return false
}
err = s.fs.MkdirAll(s.x11SocketDir, 0o700)
err = s.fs.MkdirAll(s.config.X11SocketDir, 0o700)
if err != nil {
s.logger.Warn(ctx, "failed to make the x11 socket dir", slog.F("dir", s.x11SocketDir), slog.Error(err))
s.logger.Warn(ctx, "failed to make the x11 socket dir", slog.F("dir", s.config.X11SocketDir), slog.Error(err))
s.metrics.x11HandlerErrors.WithLabelValues("socker_dir").Add(1)
return false
}
@@ -56,7 +57,7 @@ func (s *Server) x11Handler(ctx ssh.Context, x11 ssh.X11) bool {
return false
}
// We want to overwrite the socket so that subsequent connections will succeed.
socketPath := filepath.Join(s.x11SocketDir, fmt.Sprintf("X%d", x11.ScreenNumber))
socketPath := filepath.Join(s.config.X11SocketDir, fmt.Sprintf("X%d", x11.ScreenNumber))
err := os.Remove(socketPath)
if err != nil && !errors.Is(err, os.ErrNotExist) {
s.logger.Warn(ctx, "failed to remove existing X11 socket", slog.Error(err))
@@ -141,7 +142,7 @@ func addXauthEntry(ctx context.Context, fs afero.Fs, host string, display string
}
// Open or create the Xauthority file
file, err := fs.OpenFile(xauthPath, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0o600)
file, err := fs.OpenFile(xauthPath, os.O_RDWR|os.O_CREATE, 0o600)
if err != nil {
return xerrors.Errorf("failed to open Xauthority file: %w", err)
}
@@ -153,7 +154,105 @@ func addXauthEntry(ctx context.Context, fs afero.Fs, host string, display string
return xerrors.Errorf("failed to decode auth cookie: %w", err)
}
// Write Xauthority entry
// Read the Xauthority file and look for an existing entry for the host,
// display, and auth protocol. If an entry is found, overwrite the auth
// cookie (if it fits). Otherwise, mark the entry for deletion.
type deleteEntry struct {
start, end int
}
var deleteEntries []deleteEntry
pos := 0
updated := false
for {
entry, err := readXauthEntry(file)
if err != nil {
if errors.Is(err, io.EOF) {
break
}
return xerrors.Errorf("failed to read Xauthority entry: %w", err)
}
nextPos := pos + entry.Len()
cookieStartPos := nextPos - len(entry.authCookie)
if entry.family == 0x0100 && entry.address == host && entry.display == display && entry.authProtocol == authProtocol {
if !updated && len(entry.authCookie) == len(authCookieBytes) {
// Overwrite the auth cookie
_, err := file.WriteAt(authCookieBytes, int64(cookieStartPos))
if err != nil {
return xerrors.Errorf("failed to write auth cookie: %w", err)
}
updated = true
} else {
// Mark entry for deletion.
if len(deleteEntries) > 0 && deleteEntries[len(deleteEntries)-1].end == pos {
deleteEntries[len(deleteEntries)-1].end = nextPos
} else {
deleteEntries = append(deleteEntries, deleteEntry{
start: pos,
end: nextPos,
})
}
}
}
pos = nextPos
}
// In case the magic cookie changed, or we've previously bloated the
// Xauthority file, we may have to delete entries.
if len(deleteEntries) > 0 {
// Read the entire file into memory. This is not ideal, but it's the
// simplest way to delete entries from the middle of the file. The
// Xauthority file is small, so this should be fine.
_, err = file.Seek(0, io.SeekStart)
if err != nil {
return xerrors.Errorf("failed to seek Xauthority file: %w", err)
}
data, err := io.ReadAll(file)
if err != nil {
return xerrors.Errorf("failed to read Xauthority file: %w", err)
}
// Delete the entries in reverse order.
for i := len(deleteEntries) - 1; i >= 0; i-- {
entry := deleteEntries[i]
// Safety check: ensure the entry is still there.
if entry.start > len(data) || entry.end > len(data) {
continue
}
data = append(data[:entry.start], data[entry.end:]...)
}
// Write the data back to the file.
_, err = file.Seek(0, io.SeekStart)
if err != nil {
return xerrors.Errorf("failed to seek Xauthority file: %w", err)
}
_, err = file.Write(data)
if err != nil {
return xerrors.Errorf("failed to write Xauthority file: %w", err)
}
// Truncate the file.
err = file.Truncate(int64(len(data)))
if err != nil {
return xerrors.Errorf("failed to truncate Xauthority file: %w", err)
}
}
// Return if we've already updated the entry.
if updated {
return nil
}
// Ensure we're at the end (append).
_, err = file.Seek(0, io.SeekEnd)
if err != nil {
return xerrors.Errorf("failed to seek Xauthority file: %w", err)
}
// Append Xauthority entry.
family := uint16(0x0100) // FamilyLocal
err = binary.Write(file, binary.BigEndian, family)
if err != nil {
@@ -198,3 +297,96 @@ func addXauthEntry(ctx context.Context, fs afero.Fs, host string, display string
return nil
}
// xauthEntry is an representation of an Xauthority entry.
//
// The Xauthority file format is as follows:
//
// - 16-bit family
// - 16-bit address length
// - address
// - 16-bit display length
// - display
// - 16-bit auth protocol length
// - auth protocol
// - 16-bit auth cookie length
// - auth cookie
type xauthEntry struct {
family uint16
address string
display string
authProtocol string
authCookie []byte
}
func (e xauthEntry) Len() int {
// 5 * uint16 = 10 bytes for the family/length fields.
return 2*5 + len(e.address) + len(e.display) + len(e.authProtocol) + len(e.authCookie)
}
func readXauthEntry(r io.Reader) (xauthEntry, error) {
var entry xauthEntry
// Read family
err := binary.Read(r, binary.BigEndian, &entry.family)
if err != nil {
return xauthEntry{}, xerrors.Errorf("failed to read family: %w", err)
}
// Read address
var addressLength uint16
err = binary.Read(r, binary.BigEndian, &addressLength)
if err != nil {
return xauthEntry{}, xerrors.Errorf("failed to read address length: %w", err)
}
addressBytes := make([]byte, addressLength)
_, err = r.Read(addressBytes)
if err != nil {
return xauthEntry{}, xerrors.Errorf("failed to read address: %w", err)
}
entry.address = string(addressBytes)
// Read display
var displayLength uint16
err = binary.Read(r, binary.BigEndian, &displayLength)
if err != nil {
return xauthEntry{}, xerrors.Errorf("failed to read display length: %w", err)
}
displayBytes := make([]byte, displayLength)
_, err = r.Read(displayBytes)
if err != nil {
return xauthEntry{}, xerrors.Errorf("failed to read display: %w", err)
}
entry.display = string(displayBytes)
// Read auth protocol
var authProtocolLength uint16
err = binary.Read(r, binary.BigEndian, &authProtocolLength)
if err != nil {
return xauthEntry{}, xerrors.Errorf("failed to read auth protocol length: %w", err)
}
authProtocolBytes := make([]byte, authProtocolLength)
_, err = r.Read(authProtocolBytes)
if err != nil {
return xauthEntry{}, xerrors.Errorf("failed to read auth protocol: %w", err)
}
entry.authProtocol = string(authProtocolBytes)
// Read auth cookie
var authCookieLength uint16
err = binary.Read(r, binary.BigEndian, &authCookieLength)
if err != nil {
return xauthEntry{}, xerrors.Errorf("failed to read auth cookie length: %w", err)
}
entry.authCookie = make([]byte, authCookieLength)
_, err = r.Read(entry.authCookie)
if err != nil {
return xauthEntry{}, xerrors.Errorf("failed to read auth cookie: %w", err)
}
return entry, nil
}
+254
View File
@@ -0,0 +1,254 @@
package agentssh
import (
"context"
"os"
"path/filepath"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_addXauthEntry(t *testing.T) {
t.Parallel()
type testEntry struct {
address string
display string
authProtocol string
authCookie string
}
tests := []struct {
name string
authFile []byte
wantAuthFile []byte
entries []testEntry
}{
{
name: "add entry",
authFile: nil,
wantAuthFile: []byte{
// w/unix:0 MIT-MAGIC-COOKIE-1 00
//
// 00000000: 0100 0001 7700 0130 0012 4d49 542d 4d41 ....w..0..MIT-MA
// 00000010: 4749 432d 434f 4f4b 4945 2d31 0001 00 GIC-COOKIE-1...
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x30,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0x00,
},
entries: []testEntry{
{
address: "w",
display: "0",
authProtocol: "MIT-MAGIC-COOKIE-1",
authCookie: "00",
},
},
},
{
name: "add two entries",
authFile: []byte{},
wantAuthFile: []byte{
// w/unix:0 MIT-MAGIC-COOKIE-1 00
// w/unix:1 MIT-MAGIC-COOKIE-1 11
//
// 00000000: 0100 0001 7700 0130 0012 4d49 542d 4d41 ....w..0..MIT-MA
// 00000010: 4749 432d 434f 4f4b 4945 2d31 0001 0001 GIC-COOKIE-1....
// 00000020: 0000 0177 0001 3100 124d 4954 2d4d 4147 ...w..1..MIT-MAG
// 00000030: 4943 2d43 4f4f 4b49 452d 3100 0111 IC-COOKIE-1...
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x30,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0x00,
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x31,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0x11,
},
entries: []testEntry{
{
address: "w",
display: "0",
authProtocol: "MIT-MAGIC-COOKIE-1",
authCookie: "00",
},
{
address: "w",
display: "1",
authProtocol: "MIT-MAGIC-COOKIE-1",
authCookie: "11",
},
},
},
{
name: "update entry with new auth cookie length",
authFile: []byte{
// w/unix:0 MIT-MAGIC-COOKIE-1 00
// w/unix:1 MIT-MAGIC-COOKIE-1 11
//
// 00000000: 0100 0001 7700 0130 0012 4d49 542d 4d41 ....w..0..MIT-MA
// 00000010: 4749 432d 434f 4f4b 4945 2d31 0001 0001 GIC-COOKIE-1....
// 00000020: 0000 0177 0001 3100 124d 4954 2d4d 4147 ...w..1..MIT-MAG
// 00000030: 4943 2d43 4f4f 4b49 452d 3100 0111 IC-COOKIE-1...
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x30,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0x00,
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x31,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0x11,
},
wantAuthFile: []byte{
// The order changed, due to new length of auth cookie resulting
// in remove + append, we verify that the implementation is
// behaving as expected (changing the order is not a requirement,
// simply an implementation detail).
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x31,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0x11,
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x30,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x02, 0xff, 0xff,
},
entries: []testEntry{
{
address: "w",
display: "0",
authProtocol: "MIT-MAGIC-COOKIE-1",
authCookie: "ffff",
},
},
},
{
name: "update entry",
authFile: []byte{
// 00000000: 0100 0001 7700 0130 0012 4d49 542d 4d41 ....w..0..MIT-MA
// 00000010: 4749 432d 434f 4f4b 4945 2d31 0001 0001 GIC-COOKIE-1....
// 00000020: 0000 0177 0001 3100 124d 4954 2d4d 4147 ...w..1..MIT-MAG
// 00000030: 4943 2d43 4f4f 4b49 452d 3100 0111 IC-COOKIE-1...
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x30,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0x00,
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x31,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0x11,
},
wantAuthFile: []byte{
// 00000000: 0100 0001 7700 0130 0012 4d49 542d 4d41 ....w..0..MIT-MA
// 00000010: 4749 432d 434f 4f4b 4945 2d31 0001 0001 GIC-COOKIE-1....
// 00000020: 0000 0177 0001 3100 124d 4954 2d4d 4147 ...w..1..MIT-MAG
// 00000030: 4943 2d43 4f4f 4b49 452d 3100 0111 IC-COOKIE-1...
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x30,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0xff,
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x31,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x01, 0x11,
},
entries: []testEntry{
{
address: "w",
display: "0",
authProtocol: "MIT-MAGIC-COOKIE-1",
authCookie: "ff",
},
},
},
{
name: "clean up old entries",
authFile: []byte{
// w/unix:0 MIT-MAGIC-COOKIE-1 80507df050756cdefa504b65adb3bcfb
// w/unix:0 MIT-MAGIC-COOKIE-1 267b37f6cbc11b97beb826bb1aab8570
// w/unix:0 MIT-MAGIC-COOKIE-1 516e22e2b11d1bd0115dff09c028ca5c
//
// 00000000: 0100 0001 7700 0130 0012 4d49 542d 4d41 ....w..0..MIT-MA
// 00000010: 4749 432d 434f 4f4b 4945 2d31 0010 8050 GIC-COOKIE-1...P
// 00000020: 7df0 5075 6cde fa50 4b65 adb3 bcfb 0100 }.Pul..PKe......
// 00000030: 0001 7700 0130 0012 4d49 542d 4d41 4749 ..w..0..MIT-MAGI
// 00000040: 432d 434f 4f4b 4945 2d31 0010 267b 37f6 C-COOKIE-1..&{7.
// 00000050: cbc1 1b97 beb8 26bb 1aab 8570 0100 0001 ......&....p....
// 00000060: 7700 0130 0012 4d49 542d 4d41 4749 432d w..0..MIT-MAGIC-
// 00000070: 434f 4f4b 4945 2d31 0010 516e 22e2 b11d COOKIE-1..Qn"...
// 00000080: 1bd0 115d ff09 c028 ca5c ...]...(.\
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x30,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x10, 0x80, 0x50,
0x7d, 0xf0, 0x50, 0x75, 0x6c, 0xde, 0xfa, 0x50,
0x4b, 0x65, 0xad, 0xb3, 0xbc, 0xfb, 0x01, 0x00,
0x00, 0x01, 0x77, 0x00, 0x01, 0x30, 0x00, 0x12,
0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41, 0x47, 0x49,
0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b, 0x49, 0x45,
0x2d, 0x31, 0x00, 0x10, 0x26, 0x7b, 0x37, 0xf6,
0xcb, 0xc1, 0x1b, 0x97, 0xbe, 0xb8, 0x26, 0xbb,
0x1a, 0xab, 0x85, 0x70, 0x01, 0x00, 0x00, 0x01,
0x77, 0x00, 0x01, 0x30, 0x00, 0x12, 0x4d, 0x49,
0x54, 0x2d, 0x4d, 0x41, 0x47, 0x49, 0x43, 0x2d,
0x43, 0x4f, 0x4f, 0x4b, 0x49, 0x45, 0x2d, 0x31,
0x00, 0x10, 0x51, 0x6e, 0x22, 0xe2, 0xb1, 0x1d,
0x1b, 0xd0, 0x11, 0x5d, 0xff, 0x09, 0xc0, 0x28,
0xca, 0x5c,
},
wantAuthFile: []byte{
// w/unix:0 MIT-MAGIC-COOKIE-1 516e5bc892b7162b844abd1fc1a7c16e
//
// 00000000: 0100 0001 7700 0130 0012 4d49 542d 4d41 ....w..0..MIT-MA
// 00000010: 4749 432d 434f 4f4b 4945 2d31 0010 516e GIC-COOKIE-1..Qn
// 00000020: 5bc8 92b7 162b 844a bd1f c1a7 c16e [....+.J.....n
0x01, 0x00, 0x00, 0x01, 0x77, 0x00, 0x01, 0x30,
0x00, 0x12, 0x4d, 0x49, 0x54, 0x2d, 0x4d, 0x41,
0x47, 0x49, 0x43, 0x2d, 0x43, 0x4f, 0x4f, 0x4b,
0x49, 0x45, 0x2d, 0x31, 0x00, 0x10, 0x51, 0x6e,
0x5b, 0xc8, 0x92, 0xb7, 0x16, 0x2b, 0x84, 0x4a,
0xbd, 0x1f, 0xc1, 0xa7, 0xc1, 0x6e,
},
entries: []testEntry{
{
address: "w",
display: "0",
authProtocol: "MIT-MAGIC-COOKIE-1",
authCookie: "516e5bc892b7162b844abd1fc1a7c16e",
},
},
},
}
homedir, err := os.UserHomeDir()
require.NoError(t, err)
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
fs := afero.NewMemMapFs()
if tt.authFile != nil {
err := afero.WriteFile(fs, filepath.Join(homedir, ".Xauthority"), tt.authFile, 0o600)
require.NoError(t, err)
}
for _, entry := range tt.entries {
err := addXauthEntry(context.Background(), fs, entry.address, entry.display, entry.authProtocol, entry.authCookie)
require.NoError(t, err)
}
gotAuthFile, err := afero.ReadFile(fs, filepath.Join(homedir, ".Xauthority"))
require.NoError(t, err)
if diff := cmp.Diff(tt.wantAuthFile, gotAuthFile); diff != "" {
assert.Failf(t, "addXauthEntry() mismatch", "(-want +got):\n%s", diff)
}
})
}
}
+3 -7
View File
@@ -14,13 +14,11 @@ import (
"github.com/spf13/afero"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/atomic"
gossh "golang.org/x/crypto/ssh"
"cdr.dev/slog"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent/agentssh"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/testutil"
)
@@ -34,14 +32,12 @@ func TestServer_X11(t *testing.T) {
logger := slogtest.Make(t, nil).Leveled(slog.LevelDebug)
fs := afero.NewOsFs()
dir := t.TempDir()
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), fs, 0, dir)
s, err := agentssh.NewServer(ctx, logger, prometheus.NewRegistry(), fs, &agentssh.Config{
X11SocketDir: dir,
})
require.NoError(t, err)
defer s.Close()
// The assumption is that these are set before serving SSH connections.
s.AgentToken = func() string { return "" }
s.Manifest = atomic.NewPointer(&agentsdk.Manifest{})
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
+210 -142
View File
@@ -3,164 +3,133 @@ package agenttest
import (
"context"
"io"
"net"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/exp/maps"
"golang.org/x/exp/slices"
"golang.org/x/xerrors"
"google.golang.org/protobuf/types/known/durationpb"
"storj.io/drpc"
"storj.io/drpc/drpcmux"
"storj.io/drpc/drpcserver"
"tailscale.com/tailcfg"
"cdr.dev/slog"
agentproto "github.com/coder/coder/v2/agent/proto"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
drpcsdk "github.com/coder/coder/v2/codersdk/drpc"
"github.com/coder/coder/v2/tailnet"
"github.com/coder/coder/v2/tailnet/proto"
"github.com/coder/coder/v2/testutil"
)
const statsInterval = 500 * time.Millisecond
func NewClient(t testing.TB,
logger slog.Logger,
agentID uuid.UUID,
manifest agentsdk.Manifest,
statsChan chan *agentsdk.Stats,
statsChan chan *agentproto.Stats,
coordinator tailnet.Coordinator,
) *Client {
if manifest.AgentID == uuid.Nil {
manifest.AgentID = agentID
}
coordPtr := atomic.Pointer[tailnet.Coordinator]{}
coordPtr.Store(&coordinator)
mux := drpcmux.New()
derpMapUpdates := make(chan *tailcfg.DERPMap)
drpcService := &tailnet.DRPCService{
CoordPtr: &coordPtr,
Logger: logger.Named("tailnetsvc"),
DerpMapUpdateFrequency: time.Microsecond,
DerpMapFn: func() *tailcfg.DERPMap { return <-derpMapUpdates },
}
err := proto.DRPCRegisterTailnet(mux, drpcService)
require.NoError(t, err)
mp, err := agentsdk.ProtoFromManifest(manifest)
require.NoError(t, err)
fakeAAPI := NewFakeAgentAPI(t, logger, mp, statsChan)
err = agentproto.DRPCRegisterAgent(mux, fakeAAPI)
require.NoError(t, err)
server := drpcserver.NewWithOptions(mux, drpcserver.Options{
Log: func(err error) {
if xerrors.Is(err, io.EOF) {
return
}
logger.Debug(context.Background(), "drpc server error", slog.Error(err))
},
})
return &Client{
t: t,
logger: logger.Named("client"),
agentID: agentID,
manifest: manifest,
statsChan: statsChan,
coordinator: coordinator,
derpMapUpdates: make(chan agentsdk.DERPMapUpdate),
server: server,
fakeAgentAPI: fakeAAPI,
derpMapUpdates: derpMapUpdates,
}
}
type Client struct {
t testing.TB
logger slog.Logger
agentID uuid.UUID
manifest agentsdk.Manifest
metadata map[string]agentsdk.Metadata
statsChan chan *agentsdk.Stats
coordinator tailnet.Coordinator
LastWorkspaceAgent func()
PatchWorkspaceLogs func() error
GetServiceBannerFunc func() (codersdk.ServiceBannerConfig, error)
t testing.TB
logger slog.Logger
agentID uuid.UUID
coordinator tailnet.Coordinator
server *drpcserver.Server
fakeAgentAPI *FakeAgentAPI
LastWorkspaceAgent func()
mu sync.Mutex // Protects following.
lifecycleStates []codersdk.WorkspaceAgentLifecycle
startup agentsdk.PostStartupRequest
logs []agentsdk.Log
derpMapUpdates chan agentsdk.DERPMapUpdate
mu sync.Mutex // Protects following.
logs []agentsdk.Log
derpMapUpdates chan *tailcfg.DERPMap
derpMapOnce sync.Once
}
func (c *Client) Manifest(_ context.Context) (agentsdk.Manifest, error) {
return c.manifest, nil
func (*Client) RewriteDERPMap(*tailcfg.DERPMap) {}
func (c *Client) Close() {
c.derpMapOnce.Do(func() { close(c.derpMapUpdates) })
}
func (c *Client) Listen(_ context.Context) (net.Conn, error) {
clientConn, serverConn := net.Pipe()
closed := make(chan struct{})
func (c *Client) ConnectRPC(ctx context.Context) (drpc.Conn, error) {
conn, lis := drpcsdk.MemTransportPipe()
c.LastWorkspaceAgent = func() {
_ = serverConn.Close()
_ = clientConn.Close()
<-closed
_ = conn.Close()
_ = lis.Close()
}
c.t.Cleanup(c.LastWorkspaceAgent)
serveCtx, cancel := context.WithCancel(ctx)
c.t.Cleanup(cancel)
streamID := tailnet.StreamID{
Name: "agenttest",
ID: c.agentID,
Auth: tailnet.AgentCoordinateeAuth{ID: c.agentID},
}
serveCtx = tailnet.WithStreamID(serveCtx, streamID)
go func() {
_ = c.coordinator.ServeAgent(serverConn, c.agentID, "")
close(closed)
_ = c.server.Serve(serveCtx, lis)
}()
return clientConn, nil
}
func (c *Client) ReportStats(ctx context.Context, _ slog.Logger, statsChan <-chan *agentsdk.Stats, setInterval func(time.Duration)) (io.Closer, error) {
doneCh := make(chan struct{})
ctx, cancel := context.WithCancel(ctx)
go func() {
defer close(doneCh)
setInterval(500 * time.Millisecond)
for {
select {
case <-ctx.Done():
return
case stat := <-statsChan:
select {
case c.statsChan <- stat:
case <-ctx.Done():
return
default:
// We don't want to send old stats.
continue
}
}
}
}()
return closeFunc(func() error {
cancel()
<-doneCh
close(c.statsChan)
return nil
}), nil
return conn, nil
}
func (c *Client) GetLifecycleStates() []codersdk.WorkspaceAgentLifecycle {
c.mu.Lock()
defer c.mu.Unlock()
return c.lifecycleStates
return c.fakeAgentAPI.GetLifecycleStates()
}
func (c *Client) PostLifecycle(ctx context.Context, req agentsdk.PostLifecycleRequest) error {
c.mu.Lock()
defer c.mu.Unlock()
c.lifecycleStates = append(c.lifecycleStates, req.State)
c.logger.Debug(ctx, "post lifecycle", slog.F("req", req))
return nil
}
func (c *Client) PostAppHealth(ctx context.Context, req agentsdk.PostAppHealthsRequest) error {
c.logger.Debug(ctx, "post app health", slog.F("req", req))
return nil
}
func (c *Client) GetStartup() agentsdk.PostStartupRequest {
c.mu.Lock()
defer c.mu.Unlock()
return c.startup
func (c *Client) GetStartup() <-chan *agentproto.Startup {
return c.fakeAgentAPI.startupCh
}
func (c *Client) GetMetadata() map[string]agentsdk.Metadata {
c.mu.Lock()
defer c.mu.Unlock()
return maps.Clone(c.metadata)
}
func (c *Client) PostMetadata(ctx context.Context, req agentsdk.PostMetadataRequest) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.metadata == nil {
c.metadata = make(map[string]agentsdk.Metadata)
}
for _, md := range req.Metadata {
c.metadata[md.Key] = md
c.logger.Debug(ctx, "post metadata", slog.F("key", md.Key), slog.F("md", md))
}
return nil
}
func (c *Client) PostStartup(ctx context.Context, startup agentsdk.PostStartupRequest) error {
c.mu.Lock()
defer c.mu.Unlock()
c.startup = startup
c.logger.Debug(ctx, "post startup", slog.F("req", startup))
return nil
return c.fakeAgentAPI.GetMetadata()
}
func (c *Client) GetStartupLogs() []agentsdk.Log {
@@ -169,35 +138,11 @@ func (c *Client) GetStartupLogs() []agentsdk.Log {
return c.logs
}
func (c *Client) PatchLogs(ctx context.Context, logs agentsdk.PatchLogs) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.PatchWorkspaceLogs != nil {
return c.PatchWorkspaceLogs()
}
c.logs = append(c.logs, logs.Logs...)
c.logger.Debug(ctx, "patch startup logs", slog.F("req", logs))
return nil
}
func (c *Client) SetServiceBannerFunc(f func() (codersdk.ServiceBannerConfig, error)) {
c.mu.Lock()
defer c.mu.Unlock()
c.GetServiceBannerFunc = f
c.fakeAgentAPI.SetServiceBannerFunc(f)
}
func (c *Client) GetServiceBanner(ctx context.Context) (codersdk.ServiceBannerConfig, error) {
c.mu.Lock()
defer c.mu.Unlock()
c.logger.Debug(ctx, "get service banner")
if c.GetServiceBannerFunc != nil {
return c.GetServiceBannerFunc()
}
return codersdk.ServiceBannerConfig{}, nil
}
func (c *Client) PushDERPMapUpdate(update agentsdk.DERPMapUpdate) error {
func (c *Client) PushDERPMapUpdate(update *tailcfg.DERPMap) error {
timer := time.NewTimer(testutil.WaitShort)
defer timer.Stop()
select {
@@ -209,16 +154,139 @@ func (c *Client) PushDERPMapUpdate(update agentsdk.DERPMapUpdate) error {
return nil
}
func (c *Client) DERPMapUpdates(_ context.Context) (<-chan agentsdk.DERPMapUpdate, io.Closer, error) {
closed := make(chan struct{})
return c.derpMapUpdates, closeFunc(func() error {
close(closed)
return nil
}), nil
func (c *Client) SetLogsChannel(ch chan<- *agentproto.BatchCreateLogsRequest) {
c.fakeAgentAPI.SetLogsChannel(ch)
}
type closeFunc func() error
type FakeAgentAPI struct {
sync.Mutex
t testing.TB
logger slog.Logger
func (c closeFunc) Close() error {
return c()
manifest *agentproto.Manifest
startupCh chan *agentproto.Startup
statsCh chan *agentproto.Stats
appHealthCh chan *agentproto.BatchUpdateAppHealthRequest
logsCh chan<- *agentproto.BatchCreateLogsRequest
lifecycleStates []codersdk.WorkspaceAgentLifecycle
metadata map[string]agentsdk.Metadata
getServiceBannerFunc func() (codersdk.ServiceBannerConfig, error)
}
func (f *FakeAgentAPI) GetManifest(context.Context, *agentproto.GetManifestRequest) (*agentproto.Manifest, error) {
return f.manifest, nil
}
func (f *FakeAgentAPI) SetServiceBannerFunc(fn func() (codersdk.ServiceBannerConfig, error)) {
f.Lock()
defer f.Unlock()
f.getServiceBannerFunc = fn
f.logger.Info(context.Background(), "updated ServiceBannerFunc")
}
func (f *FakeAgentAPI) GetServiceBanner(context.Context, *agentproto.GetServiceBannerRequest) (*agentproto.ServiceBanner, error) {
f.Lock()
defer f.Unlock()
if f.getServiceBannerFunc == nil {
return &agentproto.ServiceBanner{}, nil
}
sb, err := f.getServiceBannerFunc()
if err != nil {
return nil, err
}
return agentsdk.ProtoFromServiceBanner(sb), nil
}
func (f *FakeAgentAPI) UpdateStats(ctx context.Context, req *agentproto.UpdateStatsRequest) (*agentproto.UpdateStatsResponse, error) {
f.logger.Debug(ctx, "update stats called", slog.F("req", req))
// empty request is sent to get the interval; but our tests don't want empty stats requests
if req.Stats != nil {
f.statsCh <- req.Stats
}
return &agentproto.UpdateStatsResponse{ReportInterval: durationpb.New(statsInterval)}, nil
}
func (f *FakeAgentAPI) GetLifecycleStates() []codersdk.WorkspaceAgentLifecycle {
f.Lock()
defer f.Unlock()
return slices.Clone(f.lifecycleStates)
}
func (f *FakeAgentAPI) UpdateLifecycle(_ context.Context, req *agentproto.UpdateLifecycleRequest) (*agentproto.Lifecycle, error) {
f.Lock()
defer f.Unlock()
s, err := agentsdk.LifecycleStateFromProto(req.GetLifecycle().GetState())
if assert.NoError(f.t, err) {
f.lifecycleStates = append(f.lifecycleStates, s)
}
return req.GetLifecycle(), nil
}
func (f *FakeAgentAPI) BatchUpdateAppHealths(ctx context.Context, req *agentproto.BatchUpdateAppHealthRequest) (*agentproto.BatchUpdateAppHealthResponse, error) {
f.logger.Debug(ctx, "batch update app health", slog.F("req", req))
f.appHealthCh <- req
return &agentproto.BatchUpdateAppHealthResponse{}, nil
}
func (f *FakeAgentAPI) AppHealthCh() <-chan *agentproto.BatchUpdateAppHealthRequest {
return f.appHealthCh
}
func (f *FakeAgentAPI) UpdateStartup(_ context.Context, req *agentproto.UpdateStartupRequest) (*agentproto.Startup, error) {
f.startupCh <- req.GetStartup()
return req.GetStartup(), nil
}
func (f *FakeAgentAPI) GetMetadata() map[string]agentsdk.Metadata {
f.Lock()
defer f.Unlock()
return maps.Clone(f.metadata)
}
func (f *FakeAgentAPI) BatchUpdateMetadata(ctx context.Context, req *agentproto.BatchUpdateMetadataRequest) (*agentproto.BatchUpdateMetadataResponse, error) {
f.Lock()
defer f.Unlock()
if f.metadata == nil {
f.metadata = make(map[string]agentsdk.Metadata)
}
for _, md := range req.Metadata {
smd := agentsdk.MetadataFromProto(md)
f.metadata[md.Key] = smd
f.logger.Debug(ctx, "post metadata", slog.F("key", md.Key), slog.F("md", md))
}
return &agentproto.BatchUpdateMetadataResponse{}, nil
}
func (f *FakeAgentAPI) SetLogsChannel(ch chan<- *agentproto.BatchCreateLogsRequest) {
f.Lock()
defer f.Unlock()
f.logsCh = ch
}
func (f *FakeAgentAPI) BatchCreateLogs(ctx context.Context, req *agentproto.BatchCreateLogsRequest) (*agentproto.BatchCreateLogsResponse, error) {
f.logger.Info(ctx, "batch create logs called", slog.F("req", req))
f.Lock()
ch := f.logsCh
f.Unlock()
if ch != nil {
select {
case <-ctx.Done():
return nil, ctx.Err()
case ch <- req:
// ok
}
}
return &agentproto.BatchCreateLogsResponse{}, nil
}
func NewFakeAgentAPI(t testing.TB, logger slog.Logger, manifest *agentproto.Manifest, statsCh chan *agentproto.Stats) *FakeAgentAPI {
return &FakeAgentAPI{
t: t,
logger: logger.Named("FakeAgentAPI"),
manifest: manifest,
statsCh: statsCh,
startupCh: make(chan *agentproto.Startup, 100),
appHealthCh: make(chan *agentproto.BatchUpdateAppHealthRequest, 100),
}
}
+6
View File
@@ -35,7 +35,13 @@ func (a *agent) apiHandler() http.Handler {
ignorePorts: cpy,
cacheDuration: cacheDuration,
}
promHandler := PrometheusMetricsHandler(a.prometheusRegistry, a.logger)
r.Get("/api/v0/listening-ports", lp.handler)
r.Get("/debug/logs", a.HandleHTTPDebugLogs)
r.Get("/debug/magicsock", a.HandleHTTPDebugMagicsock)
r.Get("/debug/magicsock/debug-logging/{state}", a.HandleHTTPMagicsockDebugLoggingState)
r.Get("/debug/manifest", a.HandleHTTPDebugManifest)
r.Get("/debug/prometheus", promHandler.ServeHTTP)
return r
}
+16 -1
View File
@@ -26,7 +26,12 @@ type WorkspaceAppHealthReporter func(ctx context.Context)
// NewWorkspaceAppHealthReporter creates a WorkspaceAppHealthReporter that reports app health to coderd.
func NewWorkspaceAppHealthReporter(logger slog.Logger, apps []codersdk.WorkspaceApp, postWorkspaceAgentAppHealth PostWorkspaceAgentAppHealth) WorkspaceAppHealthReporter {
logger = logger.Named("apphealth")
runHealthcheckLoop := func(ctx context.Context) error {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// no need to run this loop if no apps for this workspace.
if len(apps) == 0 {
return nil
@@ -87,6 +92,7 @@ func NewWorkspaceAppHealthReporter(logger slog.Logger, apps []codersdk.Workspace
return nil
}()
if err != nil {
nowUnhealthy := false
mu.Lock()
if failures[app.ID] < int(app.Healthcheck.Threshold) {
// increment the failure count and keep status the same.
@@ -96,14 +102,21 @@ func NewWorkspaceAppHealthReporter(logger slog.Logger, apps []codersdk.Workspace
// set to unhealthy if we hit the failure threshold.
// we stop incrementing at the threshold to prevent the failure value from increasing forever.
health[app.ID] = codersdk.WorkspaceAppHealthUnhealthy
nowUnhealthy = true
}
mu.Unlock()
logger.Debug(ctx, "error checking app health",
slog.F("id", app.ID.String()),
slog.F("slug", app.Slug),
slog.F("now_unhealthy", nowUnhealthy), slog.Error(err),
)
} else {
mu.Lock()
// we only need one successful health check to be considered healthy.
health[app.ID] = codersdk.WorkspaceAppHealthHealthy
failures[app.ID] = 0
mu.Unlock()
logger.Debug(ctx, "workspace app healthy", slog.F("id", app.ID.String()), slog.F("slug", app.Slug))
}
t.Reset(time.Duration(app.Healthcheck.Interval) * time.Second)
@@ -137,7 +150,9 @@ func NewWorkspaceAppHealthReporter(logger slog.Logger, apps []codersdk.Workspace
Healths: lastHealth,
})
if err != nil {
logger.Error(ctx, "failed to report workspace app stat", slog.Error(err))
logger.Error(ctx, "failed to report workspace app health", slog.Error(err))
} else {
logger.Debug(ctx, "sent workspace app health", slog.F("health", lastHealth))
}
}
}
+56 -14
View File
@@ -4,16 +4,21 @@ import (
"context"
"net/http"
"net/http/httptest"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"cdr.dev/slog"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent"
"github.com/coder/coder/v2/agent/agenttest"
"github.com/coder/coder/v2/agent/proto"
"github.com/coder/coder/v2/coderd/httpapi"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
@@ -40,12 +45,23 @@ func TestAppHealth_Healthy(t *testing.T) {
},
Health: codersdk.WorkspaceAppHealthInitializing,
},
{
Slug: "app3",
Healthcheck: codersdk.Healthcheck{
Interval: 2,
Threshold: 1,
},
Health: codersdk.WorkspaceAppHealthInitializing,
},
}
handlers := []http.Handler{
nil,
http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
httpapi.Write(r.Context(), w, http.StatusOK, nil)
}),
http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
httpapi.Write(r.Context(), w, http.StatusOK, nil)
}),
}
getApps, closeFn := setupAppReporter(ctx, t, apps, handlers)
defer closeFn()
@@ -58,7 +74,7 @@ func TestAppHealth_Healthy(t *testing.T) {
return false
}
return apps[1].Health == codersdk.WorkspaceAppHealthHealthy
return apps[1].Health == codersdk.WorkspaceAppHealthHealthy && apps[2].Health == codersdk.WorkspaceAppHealthHealthy
}, testutil.WaitLong, testutil.IntervalSlow)
}
@@ -163,6 +179,12 @@ func TestAppHealth_NotSpamming(t *testing.T) {
func setupAppReporter(ctx context.Context, t *testing.T, apps []codersdk.WorkspaceApp, handlers []http.Handler) (agent.WorkspaceAgentApps, func()) {
closers := []func(){}
for i, app := range apps {
if app.ID == uuid.Nil {
app.ID = uuid.New()
apps[i] = app
}
}
for i, handler := range handlers {
if handler == nil {
continue
@@ -181,23 +203,43 @@ func setupAppReporter(ctx context.Context, t *testing.T, apps []codersdk.Workspa
var newApps []codersdk.WorkspaceApp
return append(newApps, apps...), nil
}
postWorkspaceAgentAppHealth := func(_ context.Context, req agentsdk.PostAppHealthsRequest) error {
mu.Lock()
for id, health := range req.Healths {
for i, app := range apps {
if app.ID != id {
continue
// We don't care about manifest or stats in this test since it's not using
// a full agent and these RPCs won't get called.
//
// We use a proper fake agent API so we can test the conversion code and the
// request code as well. Before we were bypassing these by using a custom
// post function.
fakeAAPI := agenttest.NewFakeAgentAPI(t, slogtest.Make(t, nil), nil, nil)
// Process events from the channel and update the health of the apps.
go func() {
appHealthCh := fakeAAPI.AppHealthCh()
for {
select {
case <-ctx.Done():
return
case req := <-appHealthCh:
mu.Lock()
for _, update := range req.Updates {
updateID, err := uuid.FromBytes(update.Id)
assert.NoError(t, err)
updateHealth := codersdk.WorkspaceAppHealth(strings.ToLower(proto.AppHealth_name[int32(update.Health)]))
for i, app := range apps {
if app.ID != updateID {
continue
}
app.Health = updateHealth
apps[i] = app
}
}
app.Health = health
apps[i] = app
mu.Unlock()
}
}
mu.Unlock()
}()
return nil
}
go agent.NewWorkspaceAppHealthReporter(slogtest.Make(t, nil).Leveled(slog.LevelDebug), apps, postWorkspaceAgentAppHealth)(ctx)
go agent.NewWorkspaceAppHealthReporter(slogtest.Make(t, nil).Leveled(slog.LevelDebug), apps, agentsdk.AppHealthPoster(fakeAAPI))(ctx)
return workspaceAgentApps, func() {
for _, closeFn := range closers {
+14 -15
View File
@@ -10,8 +10,7 @@ import (
"tailscale.com/util/clientmetric"
"cdr.dev/slog"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/agent/proto"
)
type agentMetrics struct {
@@ -53,8 +52,8 @@ func newAgentMetrics(registerer prometheus.Registerer) *agentMetrics {
}
}
func (a *agent) collectMetrics(ctx context.Context) []agentsdk.AgentMetric {
var collected []agentsdk.AgentMetric
func (a *agent) collectMetrics(ctx context.Context) []*proto.Stats_Metric {
var collected []*proto.Stats_Metric
// Tailscale internal metrics
metrics := clientmetric.Metrics()
@@ -63,7 +62,7 @@ func (a *agent) collectMetrics(ctx context.Context) []agentsdk.AgentMetric {
continue
}
collected = append(collected, agentsdk.AgentMetric{
collected = append(collected, &proto.Stats_Metric{
Name: m.Name(),
Type: asMetricType(m.Type()),
Value: float64(m.Value()),
@@ -81,16 +80,16 @@ func (a *agent) collectMetrics(ctx context.Context) []agentsdk.AgentMetric {
labels := toAgentMetricLabels(metric.Label)
if metric.Counter != nil {
collected = append(collected, agentsdk.AgentMetric{
collected = append(collected, &proto.Stats_Metric{
Name: metricFamily.GetName(),
Type: agentsdk.AgentMetricTypeCounter,
Type: proto.Stats_Metric_COUNTER,
Value: metric.Counter.GetValue(),
Labels: labels,
})
} else if metric.Gauge != nil {
collected = append(collected, agentsdk.AgentMetric{
collected = append(collected, &proto.Stats_Metric{
Name: metricFamily.GetName(),
Type: agentsdk.AgentMetricTypeGauge,
Type: proto.Stats_Metric_GAUGE,
Value: metric.Gauge.GetValue(),
Labels: labels,
})
@@ -102,14 +101,14 @@ func (a *agent) collectMetrics(ctx context.Context) []agentsdk.AgentMetric {
return collected
}
func toAgentMetricLabels(metricLabels []*prompb.LabelPair) []agentsdk.AgentMetricLabel {
func toAgentMetricLabels(metricLabels []*prompb.LabelPair) []*proto.Stats_Metric_Label {
if len(metricLabels) == 0 {
return nil
}
labels := make([]agentsdk.AgentMetricLabel, 0, len(metricLabels))
labels := make([]*proto.Stats_Metric_Label, 0, len(metricLabels))
for _, metricLabel := range metricLabels {
labels = append(labels, agentsdk.AgentMetricLabel{
labels = append(labels, &proto.Stats_Metric_Label{
Name: metricLabel.GetName(),
Value: metricLabel.GetValue(),
})
@@ -130,12 +129,12 @@ func isIgnoredMetric(metricName string) bool {
return false
}
func asMetricType(typ clientmetric.Type) agentsdk.AgentMetricType {
func asMetricType(typ clientmetric.Type) proto.Stats_Metric_Type {
switch typ {
case clientmetric.TypeGauge:
return agentsdk.AgentMetricTypeGauge
return proto.Stats_Metric_GAUGE
case clientmetric.TypeCounter:
return agentsdk.AgentMetricTypeCounter
return proto.Stats_Metric_COUNTER
default:
panic(fmt.Sprintf("unknown metric type: %d", typ))
}
+2 -1
View File
@@ -9,6 +9,7 @@ import (
"golang.org/x/xerrors"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
)
func (lp *listeningPortsHandler) getListeningPorts() ([]codersdk.WorkspaceAgentListeningPort, error) {
@@ -32,7 +33,7 @@ func (lp *listeningPortsHandler) getListeningPorts() ([]codersdk.WorkspaceAgentL
seen := make(map[uint16]struct{}, len(tabs))
ports := []codersdk.WorkspaceAgentListeningPort{}
for _, tab := range tabs {
if tab.LocalAddr == nil || tab.LocalAddr.Port < codersdk.WorkspaceAgentMinimumListeningPort {
if tab.LocalAddr == nil || tab.LocalAddr.Port < workspacesdk.AgentMinimumListeningPort {
continue
}
+1220 -541
View File
File diff suppressed because it is too large Load Diff
+74 -22
View File
@@ -8,7 +8,7 @@ import "google/protobuf/timestamp.proto";
import "google/protobuf/duration.proto";
message WorkspaceApp {
bytes uuid = 1;
bytes id = 1;
string url = 2;
bool external = 3;
string slug = 4;
@@ -26,12 +26,12 @@ message WorkspaceApp {
}
SharingLevel sharing_level = 10;
message HealthCheck {
message Healthcheck {
string url = 1;
int32 interval = 2;
google.protobuf.Duration interval = 2;
int32 threshold = 3;
}
HealthCheck healthcheck = 11;
Healthcheck healthcheck = 11;
enum Health {
HEALTH_UNSPECIFIED = 0;
@@ -43,11 +43,54 @@ message WorkspaceApp {
Health health = 12;
}
message WorkspaceAgentScript {
bytes log_source_id = 1;
string log_path = 2;
string script = 3;
string cron = 4;
bool run_on_start = 5;
bool run_on_stop = 6;
bool start_blocks_login = 7;
google.protobuf.Duration timeout = 8;
}
message WorkspaceAgentMetadata {
message Result {
google.protobuf.Timestamp collected_at = 1;
int64 age = 2;
string value = 3;
string error = 4;
}
Result result = 1;
message Description {
string display_name = 1;
string key = 2;
string script = 3;
google.protobuf.Duration interval = 4;
google.protobuf.Duration timeout = 5;
}
Description description = 2;
}
message Manifest {
uint32 git_auth_configs = 1;
string vs_code_port_proxy_uri = 2;
repeated WorkspaceApp apps = 3;
coder.tailnet.v2.DERPMap derp_map = 4;
bytes agent_id = 1;
string agent_name = 15;
string owner_username = 13;
bytes workspace_id = 14;
string workspace_name = 16;
uint32 git_auth_configs = 2;
map<string, string> environment_variables = 3;
string directory = 4;
string vs_code_port_proxy_uri = 5;
string motd_path = 6;
bool disable_direct_connections = 7;
bool derp_force_websockets = 8;
coder.tailnet.v2.DERPMap derp_map = 9;
repeated WorkspaceAgentScript scripts = 10;
repeated WorkspaceApp apps = 11;
repeated WorkspaceAgentMetadata.Description metadata = 12;
}
message GetManifestRequest {}
@@ -100,8 +143,14 @@ message Stats {
Type type = 2;
double value = 3;
map<string, string> labels = 4;
message Label {
string name = 1;
string value = 2;
}
repeated Label labels = 4;
}
repeated Metric metrics = 12;
}
message UpdateStatsRequest{
@@ -109,14 +158,14 @@ message UpdateStatsRequest{
}
message UpdateStatsResponse {
google.protobuf.Duration report_interval_nanoseconds = 1;
google.protobuf.Duration report_interval = 1;
}
message Lifecycle {
enum State {
STATE_UNSPECIFIED = 0;
CREATED = 1;
STARTED = 2;
STARTING = 2;
START_TIMEOUT = 3;
START_ERROR = 4;
READY = 5;
@@ -126,6 +175,7 @@ message Lifecycle {
OFF = 9;
}
State state = 1;
google.protobuf.Timestamp changed_at = 2;
}
message UpdateLifecycleRequest {
@@ -142,7 +192,7 @@ enum AppHealth {
message BatchUpdateAppHealthRequest {
message HealthUpdate {
bytes uuid = 1;
bytes id = 1;
AppHealth health = 2;
}
repeated HealthUpdate updates = 1;
@@ -153,7 +203,13 @@ message BatchUpdateAppHealthResponse {}
message Startup {
string version = 1;
string expanded_directory = 2;
repeated string subsystems = 3;
enum Subsystem {
SUBSYSTEM_UNSPECIFIED = 0;
ENVBOX = 1;
ENVBUILDER = 2;
EXECTRACE = 3;
}
repeated Subsystem subsystems = 3;
}
message UpdateStartupRequest{
@@ -162,10 +218,7 @@ message UpdateStartupRequest{
message Metadata {
string key = 1;
google.protobuf.Timestamp collected_at = 2;
int64 age = 3;
string value = 4;
string error = 5;
WorkspaceAgentMetadata.Result result = 2;
}
message BatchUpdateMetadataRequest {
@@ -190,11 +243,13 @@ message Log {
}
message BatchCreateLogsRequest {
bytes source_id = 1;
bytes log_source_id = 1;
repeated Log logs = 2;
}
message BatchCreateLogsResponse {}
message BatchCreateLogsResponse {
bool log_limit_exceeded = 1;
}
service Agent {
rpc GetManifest(GetManifestRequest) returns (Manifest);
@@ -205,7 +260,4 @@ service Agent {
rpc UpdateStartup(UpdateStartupRequest) returns (Startup);
rpc BatchUpdateMetadata(BatchUpdateMetadataRequest) returns (BatchUpdateMetadataResponse);
rpc BatchCreateLogs(BatchCreateLogsRequest) returns (BatchCreateLogsResponse);
rpc StreamDERPMaps(tailnet.v2.StreamDERPMapsRequest) returns (stream tailnet.v2.DERPMap);
rpc CoordinateTailnet(stream tailnet.v2.CoordinateRequest) returns (stream tailnet.v2.CoordinateResponse);
}
+1 -149
View File
@@ -7,7 +7,6 @@ package proto
import (
context "context"
errors "errors"
proto1 "github.com/coder/coder/v2/tailnet/proto"
protojson "google.golang.org/protobuf/encoding/protojson"
proto "google.golang.org/protobuf/proto"
drpc "storj.io/drpc"
@@ -47,8 +46,6 @@ type DRPCAgentClient interface {
UpdateStartup(ctx context.Context, in *UpdateStartupRequest) (*Startup, error)
BatchUpdateMetadata(ctx context.Context, in *BatchUpdateMetadataRequest) (*BatchUpdateMetadataResponse, error)
BatchCreateLogs(ctx context.Context, in *BatchCreateLogsRequest) (*BatchCreateLogsResponse, error)
StreamDERPMaps(ctx context.Context, in *proto1.StreamDERPMapsRequest) (DRPCAgent_StreamDERPMapsClient, error)
CoordinateTailnet(ctx context.Context) (DRPCAgent_CoordinateTailnetClient, error)
}
type drpcAgentClient struct {
@@ -133,85 +130,6 @@ func (c *drpcAgentClient) BatchCreateLogs(ctx context.Context, in *BatchCreateLo
return out, nil
}
func (c *drpcAgentClient) StreamDERPMaps(ctx context.Context, in *proto1.StreamDERPMapsRequest) (DRPCAgent_StreamDERPMapsClient, error) {
stream, err := c.cc.NewStream(ctx, "/coder.agent.v2.Agent/StreamDERPMaps", drpcEncoding_File_agent_proto_agent_proto{})
if err != nil {
return nil, err
}
x := &drpcAgent_StreamDERPMapsClient{stream}
if err := x.MsgSend(in, drpcEncoding_File_agent_proto_agent_proto{}); err != nil {
return nil, err
}
if err := x.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
type DRPCAgent_StreamDERPMapsClient interface {
drpc.Stream
Recv() (*proto1.DERPMap, error)
}
type drpcAgent_StreamDERPMapsClient struct {
drpc.Stream
}
func (x *drpcAgent_StreamDERPMapsClient) GetStream() drpc.Stream {
return x.Stream
}
func (x *drpcAgent_StreamDERPMapsClient) Recv() (*proto1.DERPMap, error) {
m := new(proto1.DERPMap)
if err := x.MsgRecv(m, drpcEncoding_File_agent_proto_agent_proto{}); err != nil {
return nil, err
}
return m, nil
}
func (x *drpcAgent_StreamDERPMapsClient) RecvMsg(m *proto1.DERPMap) error {
return x.MsgRecv(m, drpcEncoding_File_agent_proto_agent_proto{})
}
func (c *drpcAgentClient) CoordinateTailnet(ctx context.Context) (DRPCAgent_CoordinateTailnetClient, error) {
stream, err := c.cc.NewStream(ctx, "/coder.agent.v2.Agent/CoordinateTailnet", drpcEncoding_File_agent_proto_agent_proto{})
if err != nil {
return nil, err
}
x := &drpcAgent_CoordinateTailnetClient{stream}
return x, nil
}
type DRPCAgent_CoordinateTailnetClient interface {
drpc.Stream
Send(*proto1.CoordinateRequest) error
Recv() (*proto1.CoordinateResponse, error)
}
type drpcAgent_CoordinateTailnetClient struct {
drpc.Stream
}
func (x *drpcAgent_CoordinateTailnetClient) GetStream() drpc.Stream {
return x.Stream
}
func (x *drpcAgent_CoordinateTailnetClient) Send(m *proto1.CoordinateRequest) error {
return x.MsgSend(m, drpcEncoding_File_agent_proto_agent_proto{})
}
func (x *drpcAgent_CoordinateTailnetClient) Recv() (*proto1.CoordinateResponse, error) {
m := new(proto1.CoordinateResponse)
if err := x.MsgRecv(m, drpcEncoding_File_agent_proto_agent_proto{}); err != nil {
return nil, err
}
return m, nil
}
func (x *drpcAgent_CoordinateTailnetClient) RecvMsg(m *proto1.CoordinateResponse) error {
return x.MsgRecv(m, drpcEncoding_File_agent_proto_agent_proto{})
}
type DRPCAgentServer interface {
GetManifest(context.Context, *GetManifestRequest) (*Manifest, error)
GetServiceBanner(context.Context, *GetServiceBannerRequest) (*ServiceBanner, error)
@@ -221,8 +139,6 @@ type DRPCAgentServer interface {
UpdateStartup(context.Context, *UpdateStartupRequest) (*Startup, error)
BatchUpdateMetadata(context.Context, *BatchUpdateMetadataRequest) (*BatchUpdateMetadataResponse, error)
BatchCreateLogs(context.Context, *BatchCreateLogsRequest) (*BatchCreateLogsResponse, error)
StreamDERPMaps(*proto1.StreamDERPMapsRequest, DRPCAgent_StreamDERPMapsStream) error
CoordinateTailnet(DRPCAgent_CoordinateTailnetStream) error
}
type DRPCAgentUnimplementedServer struct{}
@@ -259,17 +175,9 @@ func (s *DRPCAgentUnimplementedServer) BatchCreateLogs(context.Context, *BatchCr
return nil, drpcerr.WithCode(errors.New("Unimplemented"), drpcerr.Unimplemented)
}
func (s *DRPCAgentUnimplementedServer) StreamDERPMaps(*proto1.StreamDERPMapsRequest, DRPCAgent_StreamDERPMapsStream) error {
return drpcerr.WithCode(errors.New("Unimplemented"), drpcerr.Unimplemented)
}
func (s *DRPCAgentUnimplementedServer) CoordinateTailnet(DRPCAgent_CoordinateTailnetStream) error {
return drpcerr.WithCode(errors.New("Unimplemented"), drpcerr.Unimplemented)
}
type DRPCAgentDescription struct{}
func (DRPCAgentDescription) NumMethods() int { return 10 }
func (DRPCAgentDescription) NumMethods() int { return 8 }
func (DRPCAgentDescription) Method(n int) (string, drpc.Encoding, drpc.Receiver, interface{}, bool) {
switch n {
@@ -345,23 +253,6 @@ func (DRPCAgentDescription) Method(n int) (string, drpc.Encoding, drpc.Receiver,
in1.(*BatchCreateLogsRequest),
)
}, DRPCAgentServer.BatchCreateLogs, true
case 8:
return "/coder.agent.v2.Agent/StreamDERPMaps", drpcEncoding_File_agent_proto_agent_proto{},
func(srv interface{}, ctx context.Context, in1, in2 interface{}) (drpc.Message, error) {
return nil, srv.(DRPCAgentServer).
StreamDERPMaps(
in1.(*proto1.StreamDERPMapsRequest),
&drpcAgent_StreamDERPMapsStream{in2.(drpc.Stream)},
)
}, DRPCAgentServer.StreamDERPMaps, true
case 9:
return "/coder.agent.v2.Agent/CoordinateTailnet", drpcEncoding_File_agent_proto_agent_proto{},
func(srv interface{}, ctx context.Context, in1, in2 interface{}) (drpc.Message, error) {
return nil, srv.(DRPCAgentServer).
CoordinateTailnet(
&drpcAgent_CoordinateTailnetStream{in1.(drpc.Stream)},
)
}, DRPCAgentServer.CoordinateTailnet, true
default:
return "", nil, nil, nil, false
}
@@ -498,42 +389,3 @@ func (x *drpcAgent_BatchCreateLogsStream) SendAndClose(m *BatchCreateLogsRespons
}
return x.CloseSend()
}
type DRPCAgent_StreamDERPMapsStream interface {
drpc.Stream
Send(*proto1.DERPMap) error
}
type drpcAgent_StreamDERPMapsStream struct {
drpc.Stream
}
func (x *drpcAgent_StreamDERPMapsStream) Send(m *proto1.DERPMap) error {
return x.MsgSend(m, drpcEncoding_File_agent_proto_agent_proto{})
}
type DRPCAgent_CoordinateTailnetStream interface {
drpc.Stream
Send(*proto1.CoordinateResponse) error
Recv() (*proto1.CoordinateRequest, error)
}
type drpcAgent_CoordinateTailnetStream struct {
drpc.Stream
}
func (x *drpcAgent_CoordinateTailnetStream) Send(m *proto1.CoordinateResponse) error {
return x.MsgSend(m, drpcEncoding_File_agent_proto_agent_proto{})
}
func (x *drpcAgent_CoordinateTailnetStream) Recv() (*proto1.CoordinateRequest, error) {
m := new(proto1.CoordinateRequest)
if err := x.MsgRecv(m, drpcEncoding_File_agent_proto_agent_proto{}); err != nil {
return nil, err
}
return m, nil
}
func (x *drpcAgent_CoordinateTailnetStream) RecvMsg(m *proto1.CoordinateRequest) error {
return x.MsgRecv(m, drpcEncoding_File_agent_proto_agent_proto{})
}
+26
View File
@@ -0,0 +1,26 @@
package proto
func LabelsEqual(a, b []*Stats_Metric_Label) bool {
am := make(map[string]string, len(a))
for _, lbl := range a {
v := lbl.GetValue()
if v == "" {
// Prometheus considers empty labels as equivalent to being absent
continue
}
am[lbl.GetName()] = lbl.GetValue()
}
lenB := 0
for _, lbl := range b {
v := lbl.GetValue()
if v == "" {
// Prometheus considers empty labels as equivalent to being absent
continue
}
lenB++
if am[lbl.GetName()] != v {
return false
}
}
return len(am) == lenB
}
+77
View File
@@ -0,0 +1,77 @@
package proto_test
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/agent/proto"
)
func TestLabelsEqual(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
a []*proto.Stats_Metric_Label
b []*proto.Stats_Metric_Label
eq bool
}{
{
name: "mainlineEq",
a: []*proto.Stats_Metric_Label{
{Name: "credulity", Value: "sus"},
{Name: "color", Value: "aquamarine"},
},
b: []*proto.Stats_Metric_Label{
{Name: "credulity", Value: "sus"},
{Name: "color", Value: "aquamarine"},
},
eq: true,
},
{
name: "emptyValue",
a: []*proto.Stats_Metric_Label{
{Name: "credulity", Value: "sus"},
{Name: "color", Value: "aquamarine"},
{Name: "singularity", Value: ""},
},
b: []*proto.Stats_Metric_Label{
{Name: "credulity", Value: "sus"},
{Name: "color", Value: "aquamarine"},
},
eq: true,
},
{
name: "extra",
a: []*proto.Stats_Metric_Label{
{Name: "credulity", Value: "sus"},
{Name: "color", Value: "aquamarine"},
{Name: "opacity", Value: "seyshells"},
},
b: []*proto.Stats_Metric_Label{
{Name: "credulity", Value: "sus"},
{Name: "color", Value: "aquamarine"},
},
eq: false,
},
{
name: "different",
a: []*proto.Stats_Metric_Label{
{Name: "credulity", Value: "sus"},
{Name: "color", Value: "aquamarine"},
},
b: []*proto.Stats_Metric_Label{
{Name: "credulity", Value: "legit"},
{Name: "color", Value: "aquamarine"},
},
eq: false,
},
} {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
require.Equal(t, tc.eq, proto.LabelsEqual(tc.a, tc.b))
require.Equal(t, tc.eq, proto.LabelsEqual(tc.b, tc.a))
})
}
}
+10
View File
@@ -0,0 +1,10 @@
package proto
import (
"github.com/coder/coder/v2/tailnet/proto"
)
// CurrentVersion is the current version of the agent API. It is tied to the
// tailnet API version to avoid confusion, since agents connect to the tailnet
// API over the same websocket.
var CurrentVersion = proto.CurrentVersion
+2 -3
View File
@@ -14,8 +14,7 @@ import (
"golang.org/x/xerrors"
"cdr.dev/slog"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
"github.com/coder/coder/v2/pty"
)
@@ -197,7 +196,7 @@ func (s *ptyState) waitForStateOrContext(ctx context.Context, state State) (Stat
func readConnLoop(ctx context.Context, conn net.Conn, ptty pty.PTYCmd, metrics *prometheus.CounterVec, logger slog.Logger) {
decoder := json.NewDecoder(conn)
for {
var req codersdk.ReconnectingPTYRequest
var req workspacesdk.ReconnectingPTYRequest
err := decoder.Decode(&req)
if xerrors.Is(err, io.EOF) {
return
+7
View File
@@ -81,6 +81,13 @@ func newScreen(ctx context.Context, cmd *pty.Cmd, options *Options, logger slog.
rpty.id = hex.EncodeToString(buf)
settings := []string{
// Disable the startup message that appears for five seconds.
"startup_message off",
// Some message are hard-coded, the best we can do is set msgwait to 0
// which seems to hide them. This can happen for example if screen shows
// the version message when starting up.
"msgminwait 0",
"msgwait 0",
// Tell screen not to handle motion for xterm* terminals which allows
// scrolling the terminal via the mouse wheel or scroll bar (by default
// screen uses it to cycle through the command history). There does not
+126
View File
@@ -0,0 +1,126 @@
package agent
import (
"context"
"sync"
"time"
"golang.org/x/xerrors"
"tailscale.com/types/netlogtype"
"cdr.dev/slog"
"github.com/coder/coder/v2/agent/proto"
)
const maxConns = 2048
type networkStatsSource interface {
SetConnStatsCallback(maxPeriod time.Duration, maxConns int, dump func(start, end time.Time, virtual, physical map[netlogtype.Connection]netlogtype.Counts))
}
type statsCollector interface {
Collect(ctx context.Context, networkStats map[netlogtype.Connection]netlogtype.Counts) *proto.Stats
}
type statsDest interface {
UpdateStats(ctx context.Context, req *proto.UpdateStatsRequest) (*proto.UpdateStatsResponse, error)
}
// statsReporter is a subcomponent of the agent that handles registering the stats callback on the
// networkStatsSource (tailnet.Conn in prod), handling the callback, calling back to the
// statsCollector (agent in prod) to collect additional stats, then sending the update to the
// statsDest (agent API in prod)
type statsReporter struct {
*sync.Cond
networkStats *map[netlogtype.Connection]netlogtype.Counts
unreported bool
lastInterval time.Duration
source networkStatsSource
collector statsCollector
logger slog.Logger
}
func newStatsReporter(logger slog.Logger, source networkStatsSource, collector statsCollector) *statsReporter {
return &statsReporter{
Cond: sync.NewCond(&sync.Mutex{}),
logger: logger,
source: source,
collector: collector,
}
}
func (s *statsReporter) callback(_, _ time.Time, virtual, _ map[netlogtype.Connection]netlogtype.Counts) {
s.L.Lock()
defer s.L.Unlock()
s.logger.Debug(context.Background(), "got stats callback")
s.networkStats = &virtual
s.unreported = true
s.Broadcast()
}
// reportLoop programs the source (tailnet.Conn) to send it stats via the
// callback, then reports them to the dest.
//
// It's intended to be called within the larger retry loop that establishes a
// connection to the agent API, then passes that connection to go routines like
// this that use it. There is no retry and we fail on the first error since
// this will be inside a larger retry loop.
func (s *statsReporter) reportLoop(ctx context.Context, dest statsDest) error {
// send an initial, blank report to get the interval
resp, err := dest.UpdateStats(ctx, &proto.UpdateStatsRequest{})
if err != nil {
return xerrors.Errorf("initial update: %w", err)
}
s.lastInterval = resp.ReportInterval.AsDuration()
s.source.SetConnStatsCallback(s.lastInterval, maxConns, s.callback)
// use a separate goroutine to monitor the context so that we notice immediately, rather than
// waiting for the next callback (which might never come if we are closing!)
ctxDone := false
go func() {
<-ctx.Done()
s.L.Lock()
defer s.L.Unlock()
ctxDone = true
s.Broadcast()
}()
defer s.logger.Debug(ctx, "reportLoop exiting")
s.L.Lock()
defer s.L.Unlock()
for {
for !s.unreported && !ctxDone {
s.Wait()
}
if ctxDone {
return nil
}
networkStats := *s.networkStats
s.unreported = false
if err = s.reportLocked(ctx, dest, networkStats); err != nil {
return xerrors.Errorf("report stats: %w", err)
}
}
}
func (s *statsReporter) reportLocked(
ctx context.Context, dest statsDest, networkStats map[netlogtype.Connection]netlogtype.Counts,
) error {
// here we want to do our collecting/reporting while it is unlocked, but then relock
// when we return to reportLoop.
s.L.Unlock()
defer s.L.Lock()
stats := s.collector.Collect(ctx, networkStats)
resp, err := dest.UpdateStats(ctx, &proto.UpdateStatsRequest{Stats: stats})
if err != nil {
return err
}
interval := resp.GetReportInterval().AsDuration()
if interval != s.lastInterval {
s.logger.Info(ctx, "new stats report interval", slog.F("interval", interval))
s.lastInterval = interval
s.source.SetConnStatsCallback(s.lastInterval, maxConns, s.callback)
}
return nil
}
+271
View File
@@ -0,0 +1,271 @@
package agent
import (
"bytes"
"context"
"encoding/json"
"io"
"net/netip"
"sync"
"testing"
"time"
"github.com/stretchr/testify/require"
"google.golang.org/protobuf/types/known/durationpb"
"tailscale.com/types/ipproto"
"tailscale.com/types/netlogtype"
"cdr.dev/slog"
"cdr.dev/slog/sloggers/slogjson"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/agent/proto"
"github.com/coder/coder/v2/testutil"
)
func TestStatsReporter(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitShort)
logger := slogtest.Make(t, nil).Leveled(slog.LevelDebug)
fSource := newFakeNetworkStatsSource(ctx, t)
fCollector := newFakeCollector(t)
fDest := newFakeStatsDest()
uut := newStatsReporter(logger, fSource, fCollector)
loopErr := make(chan error, 1)
loopCtx, loopCancel := context.WithCancel(ctx)
go func() {
err := uut.reportLoop(loopCtx, fDest)
loopErr <- err
}()
// initial request to get duration
req := testutil.RequireRecvCtx(ctx, t, fDest.reqs)
require.NotNil(t, req)
require.Nil(t, req.Stats)
interval := time.Second * 34
testutil.RequireSendCtx(ctx, t, fDest.resps, &proto.UpdateStatsResponse{ReportInterval: durationpb.New(interval)})
// call to source to set the callback and interval
gotInterval := testutil.RequireRecvCtx(ctx, t, fSource.period)
require.Equal(t, interval, gotInterval)
// callback returning netstats
netStats := map[netlogtype.Connection]netlogtype.Counts{
{
Proto: ipproto.TCP,
Src: netip.MustParseAddrPort("192.168.1.33:4887"),
Dst: netip.MustParseAddrPort("192.168.2.99:9999"),
}: {
TxPackets: 22,
TxBytes: 23,
RxPackets: 24,
RxBytes: 25,
},
}
fSource.callback(time.Now(), time.Now(), netStats, nil)
// collector called to complete the stats
gotNetStats := testutil.RequireRecvCtx(ctx, t, fCollector.calls)
require.Equal(t, netStats, gotNetStats)
// while we are collecting the stats, send in two new netStats to simulate
// what happens if we don't keep up. Only the latest should be kept.
netStats0 := map[netlogtype.Connection]netlogtype.Counts{
{
Proto: ipproto.TCP,
Src: netip.MustParseAddrPort("192.168.1.33:4887"),
Dst: netip.MustParseAddrPort("192.168.2.99:9999"),
}: {
TxPackets: 10,
TxBytes: 10,
RxPackets: 10,
RxBytes: 10,
},
}
fSource.callback(time.Now(), time.Now(), netStats0, nil)
netStats1 := map[netlogtype.Connection]netlogtype.Counts{
{
Proto: ipproto.TCP,
Src: netip.MustParseAddrPort("192.168.1.33:4887"),
Dst: netip.MustParseAddrPort("192.168.2.99:9999"),
}: {
TxPackets: 11,
TxBytes: 11,
RxPackets: 11,
RxBytes: 11,
},
}
fSource.callback(time.Now(), time.Now(), netStats1, nil)
// complete first collection
stats := &proto.Stats{SessionCountJetbrains: 55}
testutil.RequireSendCtx(ctx, t, fCollector.stats, stats)
// destination called to report the first stats
update := testutil.RequireRecvCtx(ctx, t, fDest.reqs)
require.NotNil(t, update)
require.Equal(t, stats, update.Stats)
testutil.RequireSendCtx(ctx, t, fDest.resps, &proto.UpdateStatsResponse{ReportInterval: durationpb.New(interval)})
// second update -- only netStats1 is reported
gotNetStats = testutil.RequireRecvCtx(ctx, t, fCollector.calls)
require.Equal(t, netStats1, gotNetStats)
stats = &proto.Stats{SessionCountJetbrains: 66}
testutil.RequireSendCtx(ctx, t, fCollector.stats, stats)
update = testutil.RequireRecvCtx(ctx, t, fDest.reqs)
require.NotNil(t, update)
require.Equal(t, stats, update.Stats)
interval2 := 27 * time.Second
testutil.RequireSendCtx(ctx, t, fDest.resps, &proto.UpdateStatsResponse{ReportInterval: durationpb.New(interval2)})
// set the new interval
gotInterval = testutil.RequireRecvCtx(ctx, t, fSource.period)
require.Equal(t, interval2, gotInterval)
loopCancel()
err := testutil.RequireRecvCtx(ctx, t, loopErr)
require.NoError(t, err)
}
type fakeNetworkStatsSource struct {
sync.Mutex
ctx context.Context
t testing.TB
callback func(start, end time.Time, virtual, physical map[netlogtype.Connection]netlogtype.Counts)
period chan time.Duration
}
func (f *fakeNetworkStatsSource) SetConnStatsCallback(maxPeriod time.Duration, _ int, dump func(start time.Time, end time.Time, virtual map[netlogtype.Connection]netlogtype.Counts, physical map[netlogtype.Connection]netlogtype.Counts)) {
f.Lock()
defer f.Unlock()
f.callback = dump
select {
case <-f.ctx.Done():
f.t.Error("timeout")
case f.period <- maxPeriod:
// OK
}
}
func newFakeNetworkStatsSource(ctx context.Context, t testing.TB) *fakeNetworkStatsSource {
f := &fakeNetworkStatsSource{
ctx: ctx,
t: t,
period: make(chan time.Duration),
}
return f
}
type fakeCollector struct {
t testing.TB
calls chan map[netlogtype.Connection]netlogtype.Counts
stats chan *proto.Stats
}
func (f *fakeCollector) Collect(ctx context.Context, networkStats map[netlogtype.Connection]netlogtype.Counts) *proto.Stats {
select {
case <-ctx.Done():
f.t.Error("timeout on collect")
return nil
case f.calls <- networkStats:
// ok
}
select {
case <-ctx.Done():
f.t.Error("timeout on collect")
return nil
case s := <-f.stats:
return s
}
}
func newFakeCollector(t testing.TB) *fakeCollector {
return &fakeCollector{
t: t,
calls: make(chan map[netlogtype.Connection]netlogtype.Counts),
stats: make(chan *proto.Stats),
}
}
type fakeStatsDest struct {
reqs chan *proto.UpdateStatsRequest
resps chan *proto.UpdateStatsResponse
}
func (f *fakeStatsDest) UpdateStats(ctx context.Context, req *proto.UpdateStatsRequest) (*proto.UpdateStatsResponse, error) {
select {
case <-ctx.Done():
return nil, ctx.Err()
case f.reqs <- req:
// OK
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case resp := <-f.resps:
return resp, nil
}
}
func newFakeStatsDest() *fakeStatsDest {
return &fakeStatsDest{
reqs: make(chan *proto.UpdateStatsRequest),
resps: make(chan *proto.UpdateStatsResponse),
}
}
func Test_logDebouncer(t *testing.T) {
t.Parallel()
var (
buf bytes.Buffer
logger = slog.Make(slogjson.Sink(&buf))
ctx = context.Background()
)
debouncer := &logDebouncer{
logger: logger,
messages: map[string]time.Time{},
interval: time.Minute,
}
fields := map[string]interface{}{
"field_1": float64(1),
"field_2": "2",
}
debouncer.Error(ctx, "my message", "field_1", 1, "field_2", "2")
debouncer.Warn(ctx, "another message", "field_1", 1, "field_2", "2")
// Shouldn't log this.
debouncer.Warn(ctx, "another message", "field_1", 1, "field_2", "2")
require.Len(t, debouncer.messages, 2)
type entry struct {
Msg string `json:"msg"`
Level string `json:"level"`
Fields map[string]interface{} `json:"fields"`
}
assertLog := func(msg string, level string, fields map[string]interface{}) {
line, err := buf.ReadString('\n')
require.NoError(t, err)
var e entry
err = json.Unmarshal([]byte(line), &e)
require.NoError(t, err)
require.Equal(t, msg, e.Msg)
require.Equal(t, level, e.Level)
require.Equal(t, fields, e.Fields)
}
assertLog("my message", "ERROR", fields)
assertLog("another message", "WARN", fields)
debouncer.messages["another message"] = time.Now().Add(-2 * time.Minute)
debouncer.Warn(ctx, "another message", "field_1", 1, "field_2", "2")
assertLog("another message", "WARN", fields)
// Assert nothing else was written.
_, err := buf.ReadString('\n')
require.ErrorIs(t, err, io.EOF)
}
+89
View File
@@ -0,0 +1,89 @@
package apiversion
import (
"fmt"
"strconv"
"strings"
"golang.org/x/xerrors"
)
// New returns an *APIVersion with the given major.minor and
// additional supported major versions.
func New(maj, min int) *APIVersion {
v := &APIVersion{
supportedMajor: maj,
supportedMinor: min,
additionalMajors: make([]int, 0),
}
return v
}
type APIVersion struct {
supportedMajor int
supportedMinor int
additionalMajors []int
}
func (v *APIVersion) WithBackwardCompat(majs ...int) *APIVersion {
v.additionalMajors = append(v.additionalMajors, majs[:]...)
return v
}
func (v *APIVersion) String() string {
return fmt.Sprintf("%d.%d", v.supportedMajor, v.supportedMinor)
}
// Validate validates the given version against the given constraints:
// A given major.minor version is valid iff:
// 1. The requested major version is contained within v.supportedMajors
// 2. If the requested major version is the 'current major', then
// the requested minor version must be less than or equal to the supported
// minor version.
//
// For example, given majors {1, 2} and minor 2, then:
// - 0.x is not supported,
// - 1.x is supported,
// - 2.0, 2.1, and 2.2 are supported,
// - 2.3+ is not supported.
func (v *APIVersion) Validate(version string) error {
major, minor, err := Parse(version)
if err != nil {
return err
}
if major > v.supportedMajor {
return xerrors.Errorf("server is at version %d.%d, behind requested major version %s",
v.supportedMajor, v.supportedMinor, version)
}
if major == v.supportedMajor {
if minor > v.supportedMinor {
return xerrors.Errorf("server is at version %d.%d, behind requested minor version %s",
v.supportedMajor, v.supportedMinor, version)
}
return nil
}
for _, mjr := range v.additionalMajors {
if major == mjr {
return nil
}
}
return xerrors.Errorf("version %s is no longer supported", version)
}
// Parse parses a valid major.minor version string into (major, minor).
// Both major and minor must be valid integers separated by a period '.'.
func Parse(version string) (major int, minor int, err error) {
parts := strings.Split(version, ".")
if len(parts) != 2 {
return 0, 0, xerrors.Errorf("invalid version string: %s", version)
}
major, err = strconv.Atoi(parts[0])
if err != nil {
return 0, 0, xerrors.Errorf("invalid major version: %s", version)
}
minor, err = strconv.Atoi(parts[1])
if err != nil {
return 0, 0, xerrors.Errorf("invalid minor version: %s", version)
}
return major, minor, nil
}
+90
View File
@@ -0,0 +1,90 @@
package apiversion_test
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/apiversion"
)
func TestAPIVersionValidate(t *testing.T) {
t.Parallel()
// Given
v := apiversion.New(2, 1).WithBackwardCompat(1)
for _, tc := range []struct {
name string
version string
expectedError string
}{
{
name: "OK",
version: "2.1",
},
{
name: "MinorOK",
version: "2.0",
},
{
name: "MajorOK",
version: "1.0",
},
{
name: "TooNewMinor",
version: "2.2",
expectedError: "behind requested minor version",
},
{
name: "TooNewMajor",
version: "3.1",
expectedError: "behind requested major version",
},
{
name: "Malformed0",
version: "cats",
expectedError: "invalid version string",
},
{
name: "Malformed1",
version: "cats.dogs",
expectedError: "invalid major version",
},
{
name: "Malformed2",
version: "1.dogs",
expectedError: "invalid minor version",
},
{
name: "Malformed3",
version: "1.0.1",
expectedError: "invalid version string",
},
{
name: "Malformed4",
version: "11",
expectedError: "invalid version string",
},
{
name: "TooOld",
version: "0.8",
expectedError: "no longer supported",
},
} {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
// When
err := v.Validate(tc.version)
// Then
if tc.expectedError == "" {
require.NoError(t, err)
} else {
require.ErrorContains(t, err, tc.expectedError)
}
})
}
}
+44 -54
View File
@@ -18,10 +18,8 @@ import (
"cloud.google.com/go/compute/metadata"
"golang.org/x/xerrors"
"gopkg.in/natefinch/lumberjack.v2"
"tailscale.com/util/clientmetric"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/common/expfmt"
"cdr.dev/slog"
"cdr.dev/slog/sloggers/sloghuman"
@@ -31,15 +29,16 @@ import (
"github.com/coder/coder/v2/agent/agentproc"
"github.com/coder/coder/v2/agent/reaper"
"github.com/coder/coder/v2/buildinfo"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/serpent"
)
func (r *RootCmd) workspaceAgent() *clibase.Cmd {
func (r *RootCmd) workspaceAgent() *serpent.Command {
var (
auth string
logDir string
scriptDataDir string
pprofAddress string
noReap bool
sshMaxTimeout time.Duration
@@ -50,12 +49,12 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
slogJSONPath string
slogStackdriverPath string
)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Use: "agent",
Short: `Starts the Coder workspace agent.`,
// This command isn't useful to manually execute.
Hidden: true,
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
ctx, cancel := context.WithCancel(inv.Context())
defer cancel()
@@ -124,7 +123,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
args := append(os.Args, "--no-reap")
err := reaper.ForkReap(
reaper.WithExecArgs(args...),
reaper.WithCatchSignals(InterruptSignals...),
reaper.WithCatchSignals(StopSignals...),
)
if err != nil {
logger.Error(ctx, "agent process reaper unable to fork", slog.Error(err))
@@ -143,12 +142,12 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
// Note that we don't want to handle these signals in the
// process that runs as PID 1, that's why we do this after
// the reaper forked.
ctx, stopNotify := inv.SignalNotifyContext(ctx, InterruptSignals...)
ctx, stopNotify := inv.SignalNotifyContext(ctx, StopSignals...)
defer stopNotify()
// DumpHandler does signal handling, so we call it after the
// reaper.
go DumpHandler(ctx)
go DumpHandler(ctx, "agent")
logWriter := &lumberjackWriteCloseFixer{w: &lumberjack.Logger{
Filename: filepath.Join(logDir, "coder-agent.log"),
@@ -278,12 +277,21 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
subsystems = append(subsystems, subsystem)
}
procTicker := time.NewTicker(time.Second)
defer procTicker.Stop()
environmentVariables := map[string]string{
"GIT_ASKPASS": executablePath,
}
if v, ok := os.LookupEnv(agent.EnvProcPrioMgmt); ok {
environmentVariables[agent.EnvProcPrioMgmt] = v
}
if v, ok := os.LookupEnv(agent.EnvProcOOMScore); ok {
environmentVariables[agent.EnvProcOOMScore] = v
}
agnt := agent.New(agent.Options{
Client: client,
Logger: logger,
LogDir: logDir,
ScriptDataDir: scriptDataDir,
TailnetListenPort: uint16(tailnetListenPort),
ExchangeToken: func(ctx context.Context) (string, error) {
if exchangeToken == nil {
@@ -296,13 +304,10 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
client.SetSessionToken(resp.SessionToken)
return resp.SessionToken, nil
},
EnvironmentVariables: map[string]string{
"GIT_ASKPASS": executablePath,
agent.EnvProcPrioMgmt: os.Getenv(agent.EnvProcPrioMgmt),
},
IgnorePorts: ignorePorts,
SSHMaxTimeout: sshMaxTimeout,
Subsystems: subsystems,
EnvironmentVariables: environmentVariables,
IgnorePorts: ignorePorts,
SSHMaxTimeout: sshMaxTimeout,
Subsystems: subsystems,
PrometheusRegistry: prometheusRegistry,
Syscaller: agentproc.NewSyscaller(),
@@ -311,7 +316,8 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
ModifiedProcesses: nil,
})
prometheusSrvClose := ServeHandler(ctx, logger, prometheusMetricsHandler(prometheusRegistry, logger), prometheusAddress, "prometheus")
promHandler := agent.PrometheusMetricsHandler(prometheusRegistry, logger)
prometheusSrvClose := ServeHandler(ctx, logger, promHandler, prometheusAddress, "prometheus")
defer prometheusSrvClose()
debugSrvClose := ServeHandler(ctx, logger, agnt.HTTPDebug(), debugAddress, "debug")
@@ -322,26 +328,33 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
},
}
cmd.Options = clibase.OptionSet{
cmd.Options = serpent.OptionSet{
{
Flag: "auth",
Default: "token",
Description: "Specify the authentication type to use for the agent.",
Env: "CODER_AGENT_AUTH",
Value: clibase.StringOf(&auth),
Value: serpent.StringOf(&auth),
},
{
Flag: "log-dir",
Default: os.TempDir(),
Description: "Specify the location for the agent log files.",
Env: "CODER_AGENT_LOG_DIR",
Value: clibase.StringOf(&logDir),
Value: serpent.StringOf(&logDir),
},
{
Flag: "script-data-dir",
Default: os.TempDir(),
Description: "Specify the location for storing script data.",
Env: "CODER_AGENT_SCRIPT_DATA_DIR",
Value: serpent.StringOf(&scriptDataDir),
},
{
Flag: "pprof-address",
Default: "127.0.0.1:6060",
Env: "CODER_AGENT_PPROF_ADDRESS",
Value: clibase.StringOf(&pprofAddress),
Value: serpent.StringOf(&pprofAddress),
Description: "The address to serve pprof.",
},
{
@@ -349,7 +362,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Env: "",
Description: "Do not start a process reaper.",
Value: clibase.BoolOf(&noReap),
Value: serpent.BoolOf(&noReap),
},
{
Flag: "ssh-max-timeout",
@@ -357,27 +370,27 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Default: "72h",
Env: "CODER_AGENT_SSH_MAX_TIMEOUT",
Description: "Specify the max timeout for a SSH connection, it is advisable to set it to a minimum of 60s, but no more than 72h.",
Value: clibase.DurationOf(&sshMaxTimeout),
Value: serpent.DurationOf(&sshMaxTimeout),
},
{
Flag: "tailnet-listen-port",
Default: "0",
Env: "CODER_AGENT_TAILNET_LISTEN_PORT",
Description: "Specify a static port for Tailscale to use for listening.",
Value: clibase.Int64Of(&tailnetListenPort),
Value: serpent.Int64Of(&tailnetListenPort),
},
{
Flag: "prometheus-address",
Default: "127.0.0.1:2112",
Env: "CODER_AGENT_PROMETHEUS_ADDRESS",
Value: clibase.StringOf(&prometheusAddress),
Value: serpent.StringOf(&prometheusAddress),
Description: "The bind address to serve Prometheus metrics.",
},
{
Flag: "debug-address",
Default: "127.0.0.1:2113",
Env: "CODER_AGENT_DEBUG_ADDRESS",
Value: clibase.StringOf(&debugAddress),
Value: serpent.StringOf(&debugAddress),
Description: "The bind address to serve a debug HTTP server.",
},
{
@@ -386,7 +399,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Flag: "log-human",
Env: "CODER_AGENT_LOGGING_HUMAN",
Default: "/dev/stderr",
Value: clibase.StringOf(&slogHumanPath),
Value: serpent.StringOf(&slogHumanPath),
},
{
Name: "JSON Log Location",
@@ -394,7 +407,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Flag: "log-json",
Env: "CODER_AGENT_LOGGING_JSON",
Default: "",
Value: clibase.StringOf(&slogJSONPath),
Value: serpent.StringOf(&slogJSONPath),
},
{
Name: "Stackdriver Log Location",
@@ -402,7 +415,7 @@ func (r *RootCmd) workspaceAgent() *clibase.Cmd {
Flag: "log-stackdriver",
Env: "CODER_AGENT_LOGGING_STACKDRIVER",
Default: "",
Value: clibase.StringOf(&slogStackdriverPath),
Value: serpent.StringOf(&slogStackdriverPath),
},
}
@@ -490,26 +503,3 @@ func urlPort(u string) (int, error) {
}
return -1, xerrors.Errorf("invalid port: %s", u)
}
func prometheusMetricsHandler(prometheusRegistry *prometheus.Registry, logger slog.Logger) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
// Based on: https://github.com/tailscale/tailscale/blob/280255acae604796a1113861f5a84e6fa2dc6121/ipn/localapi/localapi.go#L489
clientmetric.WritePrometheusExpositionFormat(w)
metricFamilies, err := prometheusRegistry.Gather()
if err != nil {
logger.Error(context.Background(), "Prometheus handler can't gather metric families", slog.Error(err))
return
}
for _, metricFamily := range metricFamilies {
_, err = expfmt.MetricFamilyToText(w, metricFamily)
if err != nil {
logger.Error(context.Background(), "expfmt.MetricFamilyToText failed", slog.Error(err))
return
}
}
})
}
+42 -7
View File
@@ -19,6 +19,7 @@ import (
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/dbfake"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
"github.com/coder/coder/v2/provisionersdk/proto"
"github.com/coder/coder/v2/testutil"
)
@@ -83,14 +84,16 @@ func TestWorkspaceAgent(t *testing.T) {
ctx := inv.Context()
clitest.Start(t, inv)
coderdtest.AwaitWorkspaceAgents(t, client, r.Workspace.ID)
coderdtest.NewWorkspaceAgentWaiter(t, client, r.Workspace.ID).
MatchResources(matchAgentWithVersion).Wait()
workspace, err := client.Workspace(ctx, r.Workspace.ID)
require.NoError(t, err)
resources := workspace.LatestBuild.Resources
if assert.NotEmpty(t, workspace.LatestBuild.Resources) && assert.NotEmpty(t, resources[0].Agents) {
assert.NotEmpty(t, resources[0].Agents[0].Version)
}
dialer, err := client.DialWorkspaceAgent(ctx, resources[0].Agents[0].ID, nil)
dialer, err := workspacesdk.New(client).
DialAgent(ctx, resources[0].Agents[0].ID, nil)
require.NoError(t, err)
defer dialer.Close()
require.True(t, dialer.AwaitReachable(ctx))
@@ -120,14 +123,17 @@ func TestWorkspaceAgent(t *testing.T) {
clitest.Start(t, inv)
ctx := inv.Context()
coderdtest.AwaitWorkspaceAgents(t, client, r.Workspace.ID)
coderdtest.NewWorkspaceAgentWaiter(t, client, r.Workspace.ID).
MatchResources(matchAgentWithVersion).
Wait()
workspace, err := client.Workspace(ctx, r.Workspace.ID)
require.NoError(t, err)
resources := workspace.LatestBuild.Resources
if assert.NotEmpty(t, resources) && assert.NotEmpty(t, resources[0].Agents) {
assert.NotEmpty(t, resources[0].Agents[0].Version)
}
dialer, err := client.DialWorkspaceAgent(ctx, resources[0].Agents[0].ID, nil)
dialer, err := workspacesdk.New(client).
DialAgent(ctx, resources[0].Agents[0].ID, nil)
require.NoError(t, err)
defer dialer.Close()
require.True(t, dialer.AwaitReachable(ctx))
@@ -161,14 +167,16 @@ func TestWorkspaceAgent(t *testing.T) {
)
ctx := inv.Context()
coderdtest.AwaitWorkspaceAgents(t, client, r.Workspace.ID)
coderdtest.NewWorkspaceAgentWaiter(t, client, r.Workspace.ID).
MatchResources(matchAgentWithVersion).
Wait()
workspace, err := client.Workspace(ctx, r.Workspace.ID)
require.NoError(t, err)
resources := workspace.LatestBuild.Resources
if assert.NotEmpty(t, resources) && assert.NotEmpty(t, resources[0].Agents) {
assert.NotEmpty(t, resources[0].Agents[0].Version)
}
dialer, err := client.DialWorkspaceAgent(ctx, resources[0].Agents[0].ID, nil)
dialer, err := workspacesdk.New(client).DialAgent(ctx, resources[0].Agents[0].ID, nil)
require.NoError(t, err)
defer dialer.Close()
require.True(t, dialer.AwaitReachable(ctx))
@@ -212,7 +220,8 @@ func TestWorkspaceAgent(t *testing.T) {
clitest.Start(t, inv)
resources := coderdtest.AwaitWorkspaceAgents(t, client, r.Workspace.ID)
resources := coderdtest.NewWorkspaceAgentWaiter(t, client, r.Workspace.ID).
MatchResources(matchAgentWithSubsystems).Wait()
require.Len(t, resources, 1)
require.Len(t, resources[0].Agents, 1)
require.Len(t, resources[0].Agents[0].Subsystems, 2)
@@ -221,3 +230,29 @@ func TestWorkspaceAgent(t *testing.T) {
require.Equal(t, codersdk.AgentSubsystemExectrace, resources[0].Agents[0].Subsystems[1])
})
}
func matchAgentWithVersion(rs []codersdk.WorkspaceResource) bool {
if len(rs) < 1 {
return false
}
if len(rs[0].Agents) < 1 {
return false
}
if rs[0].Agents[0].Version == "" {
return false
}
return true
}
func matchAgentWithSubsystems(rs []codersdk.WorkspaceResource) bool {
if len(rs) < 1 {
return false
}
if len(rs[0].Agents) < 1 {
return false
}
if len(rs[0].Agents[0].Subsystems) < 1 {
return false
}
return true
}
+6 -6
View File
@@ -6,22 +6,22 @@ import (
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
func (r *RootCmd) autoupdate() *clibase.Cmd {
func (r *RootCmd) autoupdate() *serpent.Command {
client := new(codersdk.Client)
cmd := &clibase.Cmd{
cmd := &serpent.Command{
Annotations: workspaceCommand,
Use: "autoupdate <workspace> <always|never>",
Short: "Toggle auto-update policy for a workspace",
Middleware: clibase.Chain(
clibase.RequireNArgs(2),
Middleware: serpent.Chain(
serpent.RequireNArgs(2),
r.InitClient(client),
),
Handler: func(inv *clibase.Invocation) error {
Handler: func(inv *serpent.Invocation) error {
policy := strings.ToLower(inv.Args[1])
err := validateAutoUpdatePolicy(policy)
if err != nil {
-80
View File
@@ -1,80 +0,0 @@
// Package clibase offers an all-in-one solution for a highly configurable CLI
// application. Within Coder, we use it for all of our subcommands, which
// demands more functionality than cobra/viber offers.
//
// The Command interface is loosely based on the chi middleware pattern and
// http.Handler/HandlerFunc.
package clibase
import (
"strings"
"golang.org/x/exp/maps"
)
// Group describes a hierarchy of groups that an option or command belongs to.
type Group struct {
Parent *Group `json:"parent,omitempty"`
Name string `json:"name,omitempty"`
YAML string `json:"yaml,omitempty"`
Description string `json:"description,omitempty"`
}
// Ancestry returns the group and all of its parents, in order.
func (g *Group) Ancestry() []Group {
if g == nil {
return nil
}
groups := []Group{*g}
for p := g.Parent; p != nil; p = p.Parent {
// Prepend to the slice so that the order is correct.
groups = append([]Group{*p}, groups...)
}
return groups
}
func (g *Group) FullName() string {
var names []string
for _, g := range g.Ancestry() {
names = append(names, g.Name)
}
return strings.Join(names, " / ")
}
// Annotations is an arbitrary key-mapping used to extend the Option and Command types.
// Its methods won't panic if the map is nil.
type Annotations map[string]string
// Mark sets a value on the annotations map, creating one
// if it doesn't exist. Mark does not mutate the original and
// returns a copy. It is suitable for chaining.
func (a Annotations) Mark(key string, value string) Annotations {
var aa Annotations
if a != nil {
aa = maps.Clone(a)
} else {
aa = make(Annotations)
}
aa[key] = value
return aa
}
// IsSet returns true if the key is set in the annotations map.
func (a Annotations) IsSet(key string) bool {
if a == nil {
return false
}
_, ok := a[key]
return ok
}
// Get retrieves a key from the map, returning false if the key is not found
// or the map is nil.
func (a Annotations) Get(key string) (string, bool) {
if a == nil {
return "", false
}
v, ok := a[key]
return v, ok
}
-621
View File
@@ -1,621 +0,0 @@
package clibase
import (
"context"
"errors"
"flag"
"fmt"
"io"
"os"
"os/signal"
"strings"
"testing"
"unicode"
"cdr.dev/slog"
"github.com/spf13/pflag"
"golang.org/x/exp/slices"
"golang.org/x/xerrors"
"gopkg.in/yaml.v3"
"github.com/coder/coder/v2/coderd/util/slice"
)
// Cmd describes an executable command.
type Cmd struct {
// Parent is the direct parent of the command.
Parent *Cmd
// Children is a list of direct descendants.
Children []*Cmd
// Use is provided in form "command [flags] [args...]".
Use string
// Aliases is a list of alternative names for the command.
Aliases []string
// Short is a one-line description of the command.
Short string
// Hidden determines whether the command should be hidden from help.
Hidden bool
// RawArgs determines whether the command should receive unparsed arguments.
// No flags are parsed when set, and the command is responsible for parsing
// its own flags.
RawArgs bool
// Long is a detailed description of the command,
// presented on its help page. It may contain examples.
Long string
Options OptionSet
Annotations Annotations
// Middleware is called before the Handler.
// Use Chain() to combine multiple middlewares.
Middleware MiddlewareFunc
Handler HandlerFunc
HelpHandler HandlerFunc
}
// AddSubcommands adds the given subcommands, setting their
// Parent field automatically.
func (c *Cmd) AddSubcommands(cmds ...*Cmd) {
for _, cmd := range cmds {
cmd.Parent = c
c.Children = append(c.Children, cmd)
}
}
// Walk calls fn for the command and all its children.
func (c *Cmd) Walk(fn func(*Cmd)) {
fn(c)
for _, child := range c.Children {
child.Parent = c
child.Walk(fn)
}
}
// PrepareAll performs initialization and linting on the command and all its children.
func (c *Cmd) PrepareAll() error {
if c.Use == "" {
return xerrors.New("command must have a Use field so that it has a name")
}
var merr error
for i := range c.Options {
opt := &c.Options[i]
if opt.Name == "" {
switch {
case opt.Flag != "":
opt.Name = opt.Flag
case opt.Env != "":
opt.Name = opt.Env
case opt.YAML != "":
opt.Name = opt.YAML
default:
merr = errors.Join(merr, xerrors.Errorf("option must have a Name, Flag, Env or YAML field"))
}
}
if opt.Description != "" {
// Enforce that description uses sentence form.
if unicode.IsLower(rune(opt.Description[0])) {
merr = errors.Join(merr, xerrors.Errorf("option %q description should start with a capital letter", opt.Name))
}
if !strings.HasSuffix(opt.Description, ".") {
merr = errors.Join(merr, xerrors.Errorf("option %q description should end with a period", opt.Name))
}
}
}
slices.SortFunc(c.Options, func(a, b Option) int {
return slice.Ascending(a.Name, b.Name)
})
slices.SortFunc(c.Children, func(a, b *Cmd) int {
return slice.Ascending(a.Name(), b.Name())
})
for _, child := range c.Children {
child.Parent = c
err := child.PrepareAll()
if err != nil {
merr = errors.Join(merr, xerrors.Errorf("command %v: %w", child.Name(), err))
}
}
return merr
}
// Name returns the first word in the Use string.
func (c *Cmd) Name() string {
return strings.Split(c.Use, " ")[0]
}
// FullName returns the full invocation name of the command,
// as seen on the command line.
func (c *Cmd) FullName() string {
var names []string
if c.Parent != nil {
names = append(names, c.Parent.FullName())
}
names = append(names, c.Name())
return strings.Join(names, " ")
}
// FullName returns usage of the command, preceded
// by the usage of its parents.
func (c *Cmd) FullUsage() string {
var uses []string
if c.Parent != nil {
uses = append(uses, c.Parent.FullName())
}
uses = append(uses, c.Use)
return strings.Join(uses, " ")
}
// FullOptions returns the options of the command and its parents.
func (c *Cmd) FullOptions() OptionSet {
var opts OptionSet
if c.Parent != nil {
opts = append(opts, c.Parent.FullOptions()...)
}
opts = append(opts, c.Options...)
return opts
}
// Invoke creates a new invocation of the command, with
// stdio discarded.
//
// The returned invocation is not live until Run() is called.
func (c *Cmd) Invoke(args ...string) *Invocation {
return &Invocation{
Command: c,
Args: args,
Stdout: io.Discard,
Stderr: io.Discard,
Stdin: strings.NewReader(""),
Logger: slog.Make(),
}
}
// Invocation represents an instance of a command being executed.
type Invocation struct {
ctx context.Context
Command *Cmd
parsedFlags *pflag.FlagSet
Args []string
// Environ is a list of environment variables. Use EnvsWithPrefix to parse
// os.Environ.
Environ Environ
Stdout io.Writer
Stderr io.Writer
Stdin io.Reader
Logger slog.Logger
Net Net
// testing
signalNotifyContext func(parent context.Context, signals ...os.Signal) (ctx context.Context, stop context.CancelFunc)
}
// WithOS returns the invocation as a main package, filling in the invocation's unset
// fields with OS defaults.
func (inv *Invocation) WithOS() *Invocation {
return inv.with(func(i *Invocation) {
i.Stdout = os.Stdout
i.Stderr = os.Stderr
i.Stdin = os.Stdin
i.Args = os.Args[1:]
i.Environ = ParseEnviron(os.Environ(), "")
i.Net = osNet{}
})
}
// WithTestSignalNotifyContext allows overriding the default implementation of SignalNotifyContext.
// This should only be used in testing.
func (inv *Invocation) WithTestSignalNotifyContext(
_ testing.TB, // ensure we only call this from tests
f func(parent context.Context, signals ...os.Signal) (ctx context.Context, stop context.CancelFunc),
) *Invocation {
return inv.with(func(i *Invocation) {
i.signalNotifyContext = f
})
}
// SignalNotifyContext is equivalent to signal.NotifyContext, but supports being overridden in
// tests.
func (inv *Invocation) SignalNotifyContext(parent context.Context, signals ...os.Signal) (ctx context.Context, stop context.CancelFunc) {
if inv.signalNotifyContext == nil {
return signal.NotifyContext(parent, signals...)
}
return inv.signalNotifyContext(parent, signals...)
}
func (inv *Invocation) WithTestParsedFlags(
_ testing.TB, // ensure we only call this from tests
parsedFlags *pflag.FlagSet,
) *Invocation {
return inv.with(func(i *Invocation) {
i.parsedFlags = parsedFlags
})
}
func (inv *Invocation) Context() context.Context {
if inv.ctx == nil {
return context.Background()
}
return inv.ctx
}
func (inv *Invocation) ParsedFlags() *pflag.FlagSet {
if inv.parsedFlags == nil {
panic("flags not parsed, has Run() been called?")
}
return inv.parsedFlags
}
type runState struct {
allArgs []string
commandDepth int
flagParseErr error
}
func copyFlagSetWithout(fs *pflag.FlagSet, without string) *pflag.FlagSet {
fs2 := pflag.NewFlagSet("", pflag.ContinueOnError)
fs2.Usage = func() {}
fs.VisitAll(func(f *pflag.Flag) {
if f.Name == without {
return
}
fs2.AddFlag(f)
})
return fs2
}
// run recursively executes the command and its children.
// allArgs is wired through the stack so that global flags can be accepted
// anywhere in the command invocation.
func (inv *Invocation) run(state *runState) error {
err := inv.Command.Options.ParseEnv(inv.Environ)
if err != nil {
return xerrors.Errorf("parsing env: %w", err)
}
// Now the fun part, argument parsing!
children := make(map[string]*Cmd)
for _, child := range inv.Command.Children {
child.Parent = inv.Command
for _, name := range append(child.Aliases, child.Name()) {
if _, ok := children[name]; ok {
return xerrors.Errorf("duplicate command name: %s", name)
}
children[name] = child
}
}
if inv.parsedFlags == nil {
inv.parsedFlags = pflag.NewFlagSet(inv.Command.Name(), pflag.ContinueOnError)
// We handle Usage ourselves.
inv.parsedFlags.Usage = func() {}
}
// If we find a duplicate flag, we want the deeper command's flag to override
// the shallow one. Unfortunately, pflag has no way to remove a flag, so we
// have to create a copy of the flagset without a value.
inv.Command.Options.FlagSet().VisitAll(func(f *pflag.Flag) {
if inv.parsedFlags.Lookup(f.Name) != nil {
inv.parsedFlags = copyFlagSetWithout(inv.parsedFlags, f.Name)
}
inv.parsedFlags.AddFlag(f)
})
var parsedArgs []string
if !inv.Command.RawArgs {
// Flag parsing will fail on intermediate commands in the command tree,
// so we check the error after looking for a child command.
state.flagParseErr = inv.parsedFlags.Parse(state.allArgs)
parsedArgs = inv.parsedFlags.Args()
}
// Set value sources for flags.
for i, opt := range inv.Command.Options {
if fl := inv.parsedFlags.Lookup(opt.Flag); fl != nil && fl.Changed {
inv.Command.Options[i].ValueSource = ValueSourceFlag
}
}
// Read YAML configs, if any.
for _, opt := range inv.Command.Options {
path, ok := opt.Value.(*YAMLConfigPath)
if !ok || path.String() == "" {
continue
}
byt, err := os.ReadFile(path.String())
if err != nil {
return xerrors.Errorf("reading yaml: %w", err)
}
var n yaml.Node
err = yaml.Unmarshal(byt, &n)
if err != nil {
return xerrors.Errorf("decoding yaml: %w", err)
}
err = inv.Command.Options.UnmarshalYAML(&n)
if err != nil {
return xerrors.Errorf("applying yaml: %w", err)
}
}
err = inv.Command.Options.SetDefaults()
if err != nil {
return xerrors.Errorf("setting defaults: %w", err)
}
// Run child command if found (next child only)
// We must do subcommand detection after flag parsing so we don't mistake flag
// values for subcommand names.
if len(parsedArgs) > state.commandDepth {
nextArg := parsedArgs[state.commandDepth]
if child, ok := children[nextArg]; ok {
child.Parent = inv.Command
inv.Command = child
state.commandDepth++
return inv.run(state)
}
}
// Flag parse errors are irrelevant for raw args commands.
if !inv.Command.RawArgs && state.flagParseErr != nil && !errors.Is(state.flagParseErr, pflag.ErrHelp) {
return xerrors.Errorf(
"parsing flags (%v) for %q: %w",
state.allArgs,
inv.Command.FullName(), state.flagParseErr,
)
}
// All options should be set. Check all required options have sources,
// meaning they were set by the user in some way (env, flag, etc).
var missing []string
for _, opt := range inv.Command.Options {
if opt.Required && opt.ValueSource == ValueSourceNone {
missing = append(missing, opt.Flag)
}
}
if len(missing) > 0 {
return xerrors.Errorf("Missing values for the required flags: %s", strings.Join(missing, ", "))
}
if inv.Command.RawArgs {
// If we're at the root command, then the name is omitted
// from the arguments, so we can just use the entire slice.
if state.commandDepth == 0 {
inv.Args = state.allArgs
} else {
argPos, err := findArg(inv.Command.Name(), state.allArgs, inv.parsedFlags)
if err != nil {
panic(err)
}
inv.Args = state.allArgs[argPos+1:]
}
} else {
// In non-raw-arg mode, we want to skip over flags.
inv.Args = parsedArgs[state.commandDepth:]
}
mw := inv.Command.Middleware
if mw == nil {
mw = Chain()
}
ctx := inv.ctx
if ctx == nil {
ctx = context.Background()
}
ctx, cancel := context.WithCancel(ctx)
defer cancel()
inv = inv.WithContext(ctx)
if inv.Command.Handler == nil || errors.Is(state.flagParseErr, pflag.ErrHelp) {
if inv.Command.HelpHandler == nil {
return xerrors.Errorf("no handler or help for command %s", inv.Command.FullName())
}
return inv.Command.HelpHandler(inv)
}
err = mw(inv.Command.Handler)(inv)
if err != nil {
return &RunCommandError{
Cmd: inv.Command,
Err: err,
}
}
return nil
}
type RunCommandError struct {
Cmd *Cmd
Err error
}
func (e *RunCommandError) Unwrap() error {
return e.Err
}
func (e *RunCommandError) Error() string {
return fmt.Sprintf("running command %q: %+v", e.Cmd.FullName(), e.Err)
}
// findArg returns the index of the first occurrence of arg in args, skipping
// over all flags.
func findArg(want string, args []string, fs *pflag.FlagSet) (int, error) {
for i := 0; i < len(args); i++ {
arg := args[i]
if !strings.HasPrefix(arg, "-") {
if arg == want {
return i, nil
}
continue
}
// This is a flag!
if strings.Contains(arg, "=") {
// The flag contains the value in the same arg, just skip.
continue
}
// We need to check if NoOptValue is set, then we should not wait
// for the next arg to be the value.
f := fs.Lookup(strings.TrimLeft(arg, "-"))
if f == nil {
return -1, xerrors.Errorf("unknown flag: %s", arg)
}
if f.NoOptDefVal != "" {
continue
}
if i == len(args)-1 {
return -1, xerrors.Errorf("flag %s requires a value", arg)
}
// Skip the value.
i++
}
return -1, xerrors.Errorf("arg %s not found", want)
}
// Run executes the command.
// If two command share a flag name, the first command wins.
//
//nolint:revive
func (inv *Invocation) Run() (err error) {
defer func() {
// Pflag is panicky, so additional context is helpful in tests.
if flag.Lookup("test.v") == nil {
return
}
if r := recover(); r != nil {
err = xerrors.Errorf("panic recovered for %s: %v", inv.Command.FullName(), r)
panic(err)
}
}()
// We close Stdin to prevent deadlocks, e.g. when the command
// has ended but an io.Copy is still reading from Stdin.
defer func() {
if inv.Stdin == nil {
return
}
rc, ok := inv.Stdin.(io.ReadCloser)
if !ok {
return
}
e := rc.Close()
err = errors.Join(err, e)
}()
err = inv.run(&runState{
allArgs: inv.Args,
})
return err
}
// WithContext returns a copy of the Invocation with the given context.
func (inv *Invocation) WithContext(ctx context.Context) *Invocation {
return inv.with(func(i *Invocation) {
i.ctx = ctx
})
}
// with returns a copy of the Invocation with the given function applied.
func (inv *Invocation) with(fn func(*Invocation)) *Invocation {
i2 := *inv
fn(&i2)
return &i2
}
// MiddlewareFunc returns the next handler in the chain,
// or nil if there are no more.
type MiddlewareFunc func(next HandlerFunc) HandlerFunc
func chain(ms ...MiddlewareFunc) MiddlewareFunc {
return MiddlewareFunc(func(next HandlerFunc) HandlerFunc {
if len(ms) > 0 {
return chain(ms[1:]...)(ms[0](next))
}
return next
})
}
// Chain returns a Handler that first calls middleware in order.
//
//nolint:revive
func Chain(ms ...MiddlewareFunc) MiddlewareFunc {
// We need to reverse the array to provide top-to-bottom execution
// order when defining a command.
reversed := make([]MiddlewareFunc, len(ms))
for i := range ms {
reversed[len(ms)-1-i] = ms[i]
}
return chain(reversed...)
}
func RequireNArgs(want int) MiddlewareFunc {
return RequireRangeArgs(want, want)
}
// RequireRangeArgs returns a Middleware that requires the number of arguments
// to be between start and end (inclusive). If end is -1, then the number of
// arguments must be at least start.
func RequireRangeArgs(start, end int) MiddlewareFunc {
if start < 0 {
panic("start must be >= 0")
}
return func(next HandlerFunc) HandlerFunc {
return func(i *Invocation) error {
got := len(i.Args)
switch {
case start == end && got != start:
switch start {
case 0:
if len(i.Command.Children) > 0 {
return xerrors.Errorf("unrecognized subcommand %q", i.Args[0])
}
return xerrors.Errorf("wanted no args but got %v %v", got, i.Args)
default:
return xerrors.Errorf(
"wanted %v args but got %v %v",
start,
got,
i.Args,
)
}
case start > 0 && end == -1:
switch {
case got < start:
return xerrors.Errorf(
"wanted at least %v args but got %v",
start,
got,
)
default:
return next(i)
}
case start > end:
panic("start must be <= end")
case got < start || got > end:
return xerrors.Errorf(
"wanted between %v and %v args but got %v",
start, end,
got,
)
default:
return next(i)
}
}
}
}
// HandlerFunc handles an Invocation of a command.
type HandlerFunc func(i *Invocation) error
-719
View File
@@ -1,719 +0,0 @@
package clibase_test
import (
"bytes"
"context"
"fmt"
"os"
"strings"
"testing"
"github.com/stretchr/testify/require"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
)
// ioBufs is the standard input, output, and error for a command.
type ioBufs struct {
Stdin bytes.Buffer
Stdout bytes.Buffer
Stderr bytes.Buffer
}
// fakeIO sets Stdin, Stdout, and Stderr to buffers.
func fakeIO(i *clibase.Invocation) *ioBufs {
var b ioBufs
i.Stdout = &b.Stdout
i.Stderr = &b.Stderr
i.Stdin = &b.Stdin
return &b
}
func TestCommand(t *testing.T) {
t.Parallel()
cmd := func() *clibase.Cmd {
var (
verbose bool
lower bool
prefix string
reqBool bool
reqStr string
)
return &clibase.Cmd{
Use: "root [subcommand]",
Options: clibase.OptionSet{
clibase.Option{
Name: "verbose",
Flag: "verbose",
Value: clibase.BoolOf(&verbose),
},
clibase.Option{
Name: "prefix",
Flag: "prefix",
Value: clibase.StringOf(&prefix),
},
},
Children: []*clibase.Cmd{
{
Use: "required-flag --req-bool=true --req-string=foo",
Short: "Example with required flags",
Options: clibase.OptionSet{
clibase.Option{
Name: "req-bool",
Flag: "req-bool",
Value: clibase.BoolOf(&reqBool),
Required: true,
},
clibase.Option{
Name: "req-string",
Flag: "req-string",
Value: clibase.Validate(clibase.StringOf(&reqStr), func(value *clibase.String) error {
ok := strings.Contains(value.String(), " ")
if !ok {
return xerrors.Errorf("string must contain a space")
}
return nil
}),
Required: true,
},
},
Handler: func(i *clibase.Invocation) error {
_, _ = i.Stdout.Write([]byte(fmt.Sprintf("%s-%t", reqStr, reqBool)))
return nil
},
},
{
Use: "toupper [word]",
Short: "Converts a word to upper case",
Middleware: clibase.Chain(
clibase.RequireNArgs(1),
),
Aliases: []string{"up"},
Options: clibase.OptionSet{
clibase.Option{
Name: "lower",
Flag: "lower",
Value: clibase.BoolOf(&lower),
},
},
Handler: func(i *clibase.Invocation) error {
_, _ = i.Stdout.Write([]byte(prefix))
w := i.Args[0]
if lower {
w = strings.ToLower(w)
} else {
w = strings.ToUpper(w)
}
_, _ = i.Stdout.Write(
[]byte(
w,
),
)
if verbose {
i.Stdout.Write([]byte("!!!"))
}
return nil
},
},
},
}
}
t.Run("SimpleOK", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke("toupper", "hello")
io := fakeIO(i)
i.Run()
require.Equal(t, "HELLO", io.Stdout.String())
})
t.Run("Alias", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"up", "hello",
)
io := fakeIO(i)
i.Run()
require.Equal(t, "HELLO", io.Stdout.String())
})
t.Run("NoSubcommand", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"na",
)
io := fakeIO(i)
err := i.Run()
require.Empty(t, io.Stdout.String())
require.Error(t, err)
})
t.Run("BadArgs", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper",
)
io := fakeIO(i)
err := i.Run()
require.Empty(t, io.Stdout.String())
require.Error(t, err)
})
t.Run("UnknownFlags", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper", "--unknown",
)
io := fakeIO(i)
err := i.Run()
require.Empty(t, io.Stdout.String())
require.Error(t, err)
})
t.Run("Verbose", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"--verbose", "toupper", "hello",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "HELLO!!!", io.Stdout.String())
})
t.Run("Verbose=", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"--verbose=true", "toupper", "hello",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "HELLO!!!", io.Stdout.String())
})
t.Run("PrefixSpace", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"--prefix", "conv: ", "toupper", "hello",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "conv: HELLO", io.Stdout.String())
})
t.Run("GlobalFlagsAnywhere", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper", "--prefix", "conv: ", "hello", "--verbose",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "conv: HELLO!!!", io.Stdout.String())
})
t.Run("LowerVerbose", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper", "--verbose", "hello", "--lower",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "hello!!!", io.Stdout.String())
})
t.Run("ParsedFlags", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"toupper", "--verbose", "hello", "--lower",
)
_ = fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t,
"true",
i.ParsedFlags().Lookup("verbose").Value.String(),
)
})
t.Run("NoDeepChild", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"root", "level", "level", "toupper", "--verbose", "hello", "--lower",
)
fio := fakeIO(i)
require.Error(t, i.Run(), fio.Stdout.String())
})
t.Run("RequiredFlagsMissing", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag",
)
fio := fakeIO(i)
err := i.Run()
require.Error(t, err, fio.Stdout.String())
require.ErrorContains(t, err, "Missing values")
})
t.Run("RequiredFlagsMissingBool", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag", "--req-string", "foo bar",
)
fio := fakeIO(i)
err := i.Run()
require.Error(t, err, fio.Stdout.String())
require.ErrorContains(t, err, "Missing values for the required flags: req-bool")
})
t.Run("RequiredFlagsMissingString", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag", "--req-bool", "true",
)
fio := fakeIO(i)
err := i.Run()
require.Error(t, err, fio.Stdout.String())
require.ErrorContains(t, err, "Missing values for the required flags: req-string")
})
t.Run("RequiredFlagsInvalid", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag", "--req-string", "nospace",
)
fio := fakeIO(i)
err := i.Run()
require.Error(t, err, fio.Stdout.String())
require.ErrorContains(t, err, "string must contain a space")
})
t.Run("RequiredFlagsOK", func(t *testing.T) {
t.Parallel()
i := cmd().Invoke(
"required-flag", "--req-bool", "true", "--req-string", "foo bar",
)
fio := fakeIO(i)
err := i.Run()
require.NoError(t, err, fio.Stdout.String())
})
}
func TestCommand_DeepNest(t *testing.T) {
t.Parallel()
cmd := &clibase.Cmd{
Use: "1",
Children: []*clibase.Cmd{
{
Use: "2",
Children: []*clibase.Cmd{
{
Use: "3",
Handler: func(i *clibase.Invocation) error {
i.Stdout.Write([]byte("3"))
return nil
},
},
},
},
},
}
inv := cmd.Invoke("2", "3")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Equal(t, "3", stdio.Stdout.String())
}
func TestCommand_FlagOverride(t *testing.T) {
t.Parallel()
var flag string
cmd := &clibase.Cmd{
Use: "1",
Options: clibase.OptionSet{
{
Name: "flag",
Flag: "f",
Value: clibase.DiscardValue,
},
},
Children: []*clibase.Cmd{
{
Use: "2",
Options: clibase.OptionSet{
{
Name: "flag",
Flag: "f",
Value: clibase.StringOf(&flag),
},
},
Handler: func(i *clibase.Invocation) error {
return nil
},
},
},
}
err := cmd.Invoke("2", "--f", "mhmm").Run()
require.NoError(t, err)
require.Equal(t, "mhmm", flag)
}
func TestCommand_MiddlewareOrder(t *testing.T) {
t.Parallel()
mw := func(letter string) clibase.MiddlewareFunc {
return func(next clibase.HandlerFunc) clibase.HandlerFunc {
return (func(i *clibase.Invocation) error {
_, _ = i.Stdout.Write([]byte(letter))
return next(i)
})
}
}
cmd := &clibase.Cmd{
Use: "toupper [word]",
Short: "Converts a word to upper case",
Middleware: clibase.Chain(
mw("A"),
mw("B"),
mw("C"),
),
Handler: (func(i *clibase.Invocation) error {
return nil
}),
}
i := cmd.Invoke(
"hello", "world",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "ABC", io.Stdout.String())
}
func TestCommand_RawArgs(t *testing.T) {
t.Parallel()
cmd := func() *clibase.Cmd {
return &clibase.Cmd{
Use: "root",
Options: clibase.OptionSet{
{
Name: "password",
Flag: "password",
Value: clibase.StringOf(new(string)),
},
},
Children: []*clibase.Cmd{
{
Use: "sushi <args...>",
Short: "Throws back raw output",
RawArgs: true,
Handler: (func(i *clibase.Invocation) error {
if v := i.ParsedFlags().Lookup("password").Value.String(); v != "codershack" {
return xerrors.Errorf("password %q is wrong!", v)
}
i.Stdout.Write([]byte(strings.Join(i.Args, " ")))
return nil
}),
},
},
}
}
t.Run("OK", func(t *testing.T) {
// Flag parsed before the raw arg command should still work.
t.Parallel()
i := cmd().Invoke(
"--password", "codershack", "sushi", "hello", "--verbose", "world",
)
io := fakeIO(i)
require.NoError(t, i.Run())
require.Equal(t, "hello --verbose world", io.Stdout.String())
})
t.Run("BadFlag", func(t *testing.T) {
// Verbose before the raw arg command should fail.
t.Parallel()
i := cmd().Invoke(
"--password", "codershack", "--verbose", "sushi", "hello", "world",
)
io := fakeIO(i)
require.Error(t, i.Run())
require.Empty(t, io.Stdout.String())
})
t.Run("NoPassword", func(t *testing.T) {
// Flag parsed before the raw arg command should still work.
t.Parallel()
i := cmd().Invoke(
"sushi", "hello", "--verbose", "world",
)
_ = fakeIO(i)
require.Error(t, i.Run())
})
}
func TestCommand_RootRaw(t *testing.T) {
t.Parallel()
cmd := &clibase.Cmd{
RawArgs: true,
Handler: func(i *clibase.Invocation) error {
i.Stdout.Write([]byte(strings.Join(i.Args, " ")))
return nil
},
}
inv := cmd.Invoke("hello", "--verbose", "--friendly")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Equal(t, "hello --verbose --friendly", stdio.Stdout.String())
}
func TestCommand_HyphenHyphen(t *testing.T) {
t.Parallel()
cmd := &clibase.Cmd{
Handler: (func(i *clibase.Invocation) error {
i.Stdout.Write([]byte(strings.Join(i.Args, " ")))
return nil
}),
}
inv := cmd.Invoke("--", "--verbose", "--friendly")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Equal(t, "--verbose --friendly", stdio.Stdout.String())
}
func TestCommand_ContextCancels(t *testing.T) {
t.Parallel()
var gotCtx context.Context
cmd := &clibase.Cmd{
Handler: (func(i *clibase.Invocation) error {
gotCtx = i.Context()
if err := gotCtx.Err(); err != nil {
return xerrors.Errorf("unexpected context error: %w", i.Context().Err())
}
return nil
}),
}
err := cmd.Invoke().Run()
require.NoError(t, err)
require.Error(t, gotCtx.Err())
}
func TestCommand_Help(t *testing.T) {
t.Parallel()
cmd := func() *clibase.Cmd {
return &clibase.Cmd{
Use: "root",
HelpHandler: (func(i *clibase.Invocation) error {
i.Stdout.Write([]byte("abdracadabra"))
return nil
}),
Handler: (func(i *clibase.Invocation) error {
return xerrors.New("should not be called")
}),
}
}
t.Run("NoHandler", func(t *testing.T) {
t.Parallel()
c := cmd()
c.HelpHandler = nil
err := c.Invoke("--help").Run()
require.Error(t, err)
})
t.Run("Long", func(t *testing.T) {
t.Parallel()
inv := cmd().Invoke("--help")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Contains(t, stdio.Stdout.String(), "abdracadabra")
})
t.Run("Short", func(t *testing.T) {
t.Parallel()
inv := cmd().Invoke("-h")
stdio := fakeIO(inv)
err := inv.Run()
require.NoError(t, err)
require.Contains(t, stdio.Stdout.String(), "abdracadabra")
})
}
func TestCommand_SliceFlags(t *testing.T) {
t.Parallel()
cmd := func(want ...string) *clibase.Cmd {
var got []string
return &clibase.Cmd{
Use: "root",
Options: clibase.OptionSet{
{
Name: "arr",
Flag: "arr",
Default: "bad,bad,bad",
Value: clibase.StringArrayOf(&got),
},
},
Handler: (func(i *clibase.Invocation) error {
require.Equal(t, want, got)
return nil
}),
}
}
err := cmd("good", "good", "good").Invoke("--arr", "good", "--arr", "good", "--arr", "good").Run()
require.NoError(t, err)
err = cmd("bad", "bad", "bad").Invoke().Run()
require.NoError(t, err)
}
func TestCommand_EmptySlice(t *testing.T) {
t.Parallel()
cmd := func(want ...string) *clibase.Cmd {
var got []string
return &clibase.Cmd{
Use: "root",
Options: clibase.OptionSet{
{
Name: "arr",
Flag: "arr",
Default: "def,def,def",
Env: "ARR",
Value: clibase.StringArrayOf(&got),
},
},
Handler: (func(i *clibase.Invocation) error {
require.Equal(t, want, got)
return nil
}),
}
}
// Base-case, uses default.
err := cmd("def", "def", "def").Invoke().Run()
require.NoError(t, err)
// Empty-env uses default, too.
inv := cmd("def", "def", "def").Invoke()
inv.Environ.Set("ARR", "")
require.NoError(t, err)
// Reset to nothing at all via flag.
inv = cmd().Invoke("--arr", "")
inv.Environ.Set("ARR", "cant see")
err = inv.Run()
require.NoError(t, err)
// Reset to a specific value with flag.
inv = cmd("great").Invoke("--arr", "great")
inv.Environ.Set("ARR", "")
err = inv.Run()
require.NoError(t, err)
}
func TestCommand_DefaultsOverride(t *testing.T) {
t.Parallel()
test := func(name string, want string, fn func(t *testing.T, inv *clibase.Invocation)) {
t.Run(name, func(t *testing.T) {
t.Parallel()
var (
got string
config clibase.YAMLConfigPath
)
cmd := &clibase.Cmd{
Options: clibase.OptionSet{
{
Name: "url",
Flag: "url",
Default: "def.com",
Env: "URL",
Value: clibase.StringOf(&got),
YAML: "url",
},
{
Name: "config",
Flag: "config",
Default: "",
Value: &config,
},
},
Handler: (func(i *clibase.Invocation) error {
_, _ = fmt.Fprintf(i.Stdout, "%s", got)
return nil
}),
}
inv := cmd.Invoke()
stdio := fakeIO(inv)
fn(t, inv)
err := inv.Run()
require.NoError(t, err)
require.Equal(t, want, stdio.Stdout.String())
})
}
test("DefaultOverNothing", "def.com", func(t *testing.T, inv *clibase.Invocation) {})
test("FlagOverDefault", "good.com", func(t *testing.T, inv *clibase.Invocation) {
inv.Args = []string{"--url", "good.com"}
})
test("EnvOverDefault", "good.com", func(t *testing.T, inv *clibase.Invocation) {
inv.Environ.Set("URL", "good.com")
})
test("FlagOverEnv", "good.com", func(t *testing.T, inv *clibase.Invocation) {
inv.Environ.Set("URL", "bad.com")
inv.Args = []string{"--url", "good.com"}
})
test("FlagOverYAML", "good.com", func(t *testing.T, inv *clibase.Invocation) {
fi, err := os.CreateTemp(t.TempDir(), "config.yaml")
require.NoError(t, err)
defer fi.Close()
_, err = fi.WriteString("url: bad.com")
require.NoError(t, err)
inv.Args = []string{"--config", fi.Name(), "--url", "good.com"}
})
test("YAMLOverDefault", "good.com", func(t *testing.T, inv *clibase.Invocation) {
fi, err := os.CreateTemp(t.TempDir(), "config.yaml")
require.NoError(t, err)
defer fi.Close()
_, err = fi.WriteString("url: good.com")
require.NoError(t, err)
inv.Args = []string{"--config", fi.Name()}
})
}
-76
View File
@@ -1,76 +0,0 @@
package clibase
import "strings"
// name returns the name of the environment variable.
func envName(line string) string {
return strings.ToUpper(
strings.SplitN(line, "=", 2)[0],
)
}
// value returns the value of the environment variable.
func envValue(line string) string {
tokens := strings.SplitN(line, "=", 2)
if len(tokens) < 2 {
return ""
}
return tokens[1]
}
// Var represents a single environment variable of form
// NAME=VALUE.
type EnvVar struct {
Name string
Value string
}
type Environ []EnvVar
func (e Environ) ToOS() []string {
var env []string
for _, v := range e {
env = append(env, v.Name+"="+v.Value)
}
return env
}
func (e Environ) Lookup(name string) (string, bool) {
for _, v := range e {
if v.Name == name {
return v.Value, true
}
}
return "", false
}
func (e Environ) Get(name string) string {
v, _ := e.Lookup(name)
return v
}
func (e *Environ) Set(name, value string) {
for i, v := range *e {
if v.Name == name {
(*e)[i].Value = value
return
}
}
*e = append(*e, EnvVar{Name: name, Value: value})
}
// ParseEnviron returns all environment variables starting with
// prefix without said prefix.
func ParseEnviron(environ []string, prefix string) Environ {
var filtered []EnvVar
for _, line := range environ {
name := envName(line)
if strings.HasPrefix(name, prefix) {
filtered = append(filtered, EnvVar{
Name: strings.TrimPrefix(name, prefix),
Value: envValue(line),
})
}
}
return filtered
}
-44
View File
@@ -1,44 +0,0 @@
package clibase_test
import (
"reflect"
"testing"
"github.com/coder/coder/v2/cli/clibase"
)
func TestFilterNamePrefix(t *testing.T) {
t.Parallel()
type args struct {
environ []string
prefix string
}
tests := []struct {
name string
args args
want clibase.Environ
}{
{"empty", args{[]string{}, "SHIRE"}, nil},
{
"ONE",
args{
[]string{
"SHIRE_BRANDYBUCK=hmm",
},
"SHIRE_",
},
[]clibase.EnvVar{
{Name: "BRANDYBUCK", Value: "hmm"},
},
},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
if got := clibase.ParseEnviron(tt.args.environ, tt.args.prefix); !reflect.DeepEqual(got, tt.want) {
t.Errorf("FilterNamePrefix() = %v, want %v", got, tt.want)
}
})
}
}
-50
View File
@@ -1,50 +0,0 @@
package clibase
import (
"net"
"strconv"
"github.com/pion/udp"
"golang.org/x/xerrors"
)
// Net abstracts CLI commands interacting with the operating system networking.
//
// At present, it covers opening local listening sockets, since doing this
// in testing is a challenge without flakes, since it's hard to pick a port we
// know a priori will be free.
type Net interface {
// Listen has the same semantics as `net.Listen` but also supports `udp`
Listen(network, address string) (net.Listener, error)
}
// osNet is an implementation that call the real OS for networking.
type osNet struct{}
func (osNet) Listen(network, address string) (net.Listener, error) {
switch network {
case "tcp", "tcp4", "tcp6", "unix", "unixpacket":
return net.Listen(network, address)
case "udp":
host, port, err := net.SplitHostPort(address)
if err != nil {
return nil, xerrors.Errorf("split %q: %w", address, err)
}
var portInt int
portInt, err = strconv.Atoi(port)
if err != nil {
return nil, xerrors.Errorf("parse port %v from %q as int: %w", port, address, err)
}
// Use pion here so that we get a stream-style net.Conn listener, instead
// of a packet-oriented connection that can read and write to multiple
// addresses.
return udp.Listen(network, &net.UDPAddr{
IP: net.ParseIP(host),
Port: portInt,
})
default:
return nil, xerrors.Errorf("unknown listen network %q", network)
}
}
-346
View File
@@ -1,346 +0,0 @@
package clibase
import (
"bytes"
"encoding/json"
"os"
"strings"
"github.com/hashicorp/go-multierror"
"github.com/spf13/pflag"
"golang.org/x/xerrors"
)
type ValueSource string
const (
ValueSourceNone ValueSource = ""
ValueSourceFlag ValueSource = "flag"
ValueSourceEnv ValueSource = "env"
ValueSourceYAML ValueSource = "yaml"
ValueSourceDefault ValueSource = "default"
)
// Option is a configuration option for a CLI application.
type Option struct {
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
// Required means this value must be set by some means. It requires
// `ValueSource != ValueSourceNone`
// If `Default` is set, then `Required` is ignored.
Required bool `json:"required,omitempty"`
// Flag is the long name of the flag used to configure this option. If unset,
// flag configuring is disabled.
Flag string `json:"flag,omitempty"`
// FlagShorthand is the one-character shorthand for the flag. If unset, no
// shorthand is used.
FlagShorthand string `json:"flag_shorthand,omitempty"`
// Env is the environment variable used to configure this option. If unset,
// environment configuring is disabled.
Env string `json:"env,omitempty"`
// YAML is the YAML key used to configure this option. If unset, YAML
// configuring is disabled.
YAML string `json:"yaml,omitempty"`
// Default is parsed into Value if set.
Default string `json:"default,omitempty"`
// Value includes the types listed in values.go.
Value pflag.Value `json:"value,omitempty"`
// Annotations enable extensions to clibase higher up in the stack. It's useful for
// help formatting and documentation generation.
Annotations Annotations `json:"annotations,omitempty"`
// Group is a group hierarchy that helps organize this option in help, configs
// and other documentation.
Group *Group `json:"group,omitempty"`
// UseInstead is a list of options that should be used instead of this one.
// The field is used to generate a deprecation warning.
UseInstead []Option `json:"use_instead,omitempty"`
Hidden bool `json:"hidden,omitempty"`
ValueSource ValueSource `json:"value_source,omitempty"`
}
// optionNoMethods is just a wrapper around Option so we can defer to the
// default json.Unmarshaler behavior.
type optionNoMethods Option
func (o *Option) UnmarshalJSON(data []byte) error {
// If an option has no values, we have no idea how to unmarshal it.
// So just discard the json data.
if o.Value == nil {
o.Value = &DiscardValue
}
return json.Unmarshal(data, (*optionNoMethods)(o))
}
func (o Option) YAMLPath() string {
if o.YAML == "" {
return ""
}
var gs []string
for _, g := range o.Group.Ancestry() {
gs = append(gs, g.YAML)
}
return strings.Join(append(gs, o.YAML), ".")
}
// OptionSet is a group of options that can be applied to a command.
type OptionSet []Option
// UnmarshalJSON implements json.Unmarshaler for OptionSets. Options have an
// interface Value type that cannot handle unmarshalling because the types cannot
// be inferred. Since it is a slice, instantiating the Options first does not
// help.
//
// However, we typically do instantiate the slice to have the correct types.
// So this unmarshaller will attempt to find the named option in the existing
// set, if it cannot, the value is discarded. If the option exists, the value
// is unmarshalled into the existing option, and replaces the existing option.
//
// The value is discarded if it's type cannot be inferred. This behavior just
// feels "safer", although it should never happen if the correct option set
// is passed in. The situation where this could occur is if a client and server
// are on different versions with different options.
func (optSet *OptionSet) UnmarshalJSON(data []byte) error {
dec := json.NewDecoder(bytes.NewBuffer(data))
// Should be a json array, so consume the starting open bracket.
t, err := dec.Token()
if err != nil {
return xerrors.Errorf("read array open bracket: %w", err)
}
if t != json.Delim('[') {
return xerrors.Errorf("expected array open bracket, got %q", t)
}
// As long as json elements exist, consume them. The counter is used for
// better errors.
var i int
OptionSetDecodeLoop:
for dec.More() {
var opt Option
// jValue is a placeholder value that allows us to capture the
// raw json for the value to attempt to unmarshal later.
var jValue jsonValue
opt.Value = &jValue
err := dec.Decode(&opt)
if err != nil {
return xerrors.Errorf("decode %d option: %w", i, err)
}
// This counter is used to contextualize errors to show which element of
// the array we failed to decode. It is only used in the error above, as
// if the above works, we can instead use the Option.Name which is more
// descriptive and useful. So increment here for the next decode.
i++
// Try to see if the option already exists in the option set.
// If it does, just update the existing option.
for optIndex, have := range *optSet {
if have.Name == opt.Name {
if jValue != nil {
err := json.Unmarshal(jValue, &(*optSet)[optIndex].Value)
if err != nil {
return xerrors.Errorf("decode option %q value: %w", have.Name, err)
}
// Set the opt's value
opt.Value = (*optSet)[optIndex].Value
} else {
// Hopefully the user passed empty values in the option set. There is no easy way
// to tell, and if we do not do this, it breaks json.Marshal if we do it again on
// this new option set.
opt.Value = (*optSet)[optIndex].Value
}
// Override the existing.
(*optSet)[optIndex] = opt
// Go to the next option to decode.
continue OptionSetDecodeLoop
}
}
// If the option doesn't exist, the value will be discarded.
// We do this because we cannot infer the type of the value.
opt.Value = DiscardValue
*optSet = append(*optSet, opt)
}
t, err = dec.Token()
if err != nil {
return xerrors.Errorf("read array close bracket: %w", err)
}
if t != json.Delim(']') {
return xerrors.Errorf("expected array close bracket, got %q", t)
}
return nil
}
// Add adds the given Options to the OptionSet.
func (optSet *OptionSet) Add(opts ...Option) {
*optSet = append(*optSet, opts...)
}
// Filter will only return options that match the given filter. (return true)
func (optSet OptionSet) Filter(filter func(opt Option) bool) OptionSet {
cpy := make(OptionSet, 0)
for _, opt := range optSet {
if filter(opt) {
cpy = append(cpy, opt)
}
}
return cpy
}
// FlagSet returns a pflag.FlagSet for the OptionSet.
func (optSet *OptionSet) FlagSet() *pflag.FlagSet {
if optSet == nil {
return &pflag.FlagSet{}
}
fs := pflag.NewFlagSet("", pflag.ContinueOnError)
for _, opt := range *optSet {
if opt.Flag == "" {
continue
}
var noOptDefValue string
{
no, ok := opt.Value.(NoOptDefValuer)
if ok {
noOptDefValue = no.NoOptDefValue()
}
}
val := opt.Value
if val == nil {
val = DiscardValue
}
fs.AddFlag(&pflag.Flag{
Name: opt.Flag,
Shorthand: opt.FlagShorthand,
Usage: opt.Description,
Value: val,
DefValue: "",
Changed: false,
Deprecated: "",
NoOptDefVal: noOptDefValue,
Hidden: opt.Hidden,
})
}
fs.Usage = func() {
_, _ = os.Stderr.WriteString("Override (*FlagSet).Usage() to print help text.\n")
}
return fs
}
// ParseEnv parses the given environment variables into the OptionSet.
// Use EnvsWithPrefix to filter out prefixes.
func (optSet *OptionSet) ParseEnv(vs []EnvVar) error {
if optSet == nil {
return nil
}
var merr *multierror.Error
// We parse environment variables first instead of using a nested loop to
// avoid N*M complexity when there are a lot of options and environment
// variables.
envs := make(map[string]string)
for _, v := range vs {
envs[v.Name] = v.Value
}
for i, opt := range *optSet {
if opt.Env == "" {
continue
}
envVal, ok := envs[opt.Env]
if !ok {
// Homebrew strips all environment variables that do not start with `HOMEBREW_`.
// This prevented using brew to invoke the Coder agent, because the environment
// variables to not get passed down.
//
// A customer wanted to use their custom tap inside a workspace, which was failing
// because the agent lacked the environment variables to authenticate with Git.
envVal, ok = envs[`HOMEBREW_`+opt.Env]
}
// Currently, empty values are treated as if the environment variable is
// unset. This behavior is technically not correct as there is now no
// way for a user to change a Default value to an empty string from
// the environment. Unfortunately, we have old configuration files
// that rely on the faulty behavior.
//
// TODO: We should remove this hack in May 2023, when deployments
// have had months to migrate to the new behavior.
if !ok || envVal == "" {
continue
}
(*optSet)[i].ValueSource = ValueSourceEnv
if err := opt.Value.Set(envVal); err != nil {
merr = multierror.Append(
merr, xerrors.Errorf("parse %q: %w", opt.Name, err),
)
}
}
return merr.ErrorOrNil()
}
// SetDefaults sets the default values for each Option, skipping values
// that already have a value source.
func (optSet *OptionSet) SetDefaults() error {
if optSet == nil {
return nil
}
var merr *multierror.Error
for i, opt := range *optSet {
// Skip values that may have already been set by the user.
if opt.ValueSource != ValueSourceNone {
continue
}
if opt.Default == "" {
continue
}
if opt.Value == nil {
merr = multierror.Append(
merr,
xerrors.Errorf(
"parse %q: no Value field set\nFull opt: %+v",
opt.Name, opt,
),
)
continue
}
(*optSet)[i].ValueSource = ValueSourceDefault
if err := opt.Value.Set(opt.Default); err != nil {
merr = multierror.Append(
merr, xerrors.Errorf("parse %q: %w", opt.Name, err),
)
}
}
return merr.ErrorOrNil()
}
// ByName returns the Option with the given name, or nil if no such option
// exists.
func (optSet *OptionSet) ByName(name string) *Option {
for i := range *optSet {
opt := &(*optSet)[i]
if opt.Name == name {
return opt
}
}
return nil
}
-391
View File
@@ -1,391 +0,0 @@
package clibase_test
import (
"bytes"
"encoding/json"
"regexp"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/codersdk"
)
func TestOptionSet_ParseFlags(t *testing.T) {
t.Parallel()
t.Run("SimpleString", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Flag: "workspace-name",
FlagShorthand: "n",
},
}
var err error
err = os.FlagSet().Parse([]string{"--workspace-name", "foo"})
require.NoError(t, err)
require.EqualValues(t, "foo", workspaceName)
err = os.FlagSet().Parse([]string{"-n", "f"})
require.NoError(t, err)
require.EqualValues(t, "f", workspaceName)
})
t.Run("StringArray", func(t *testing.T) {
t.Parallel()
var names clibase.StringArray
os := clibase.OptionSet{
clibase.Option{
Name: "name",
Value: &names,
Flag: "name",
FlagShorthand: "n",
},
}
err := os.SetDefaults()
require.NoError(t, err)
err = os.FlagSet().Parse([]string{"--name", "foo", "--name", "bar"})
require.NoError(t, err)
require.EqualValues(t, []string{"foo", "bar"}, names)
})
t.Run("ExtraFlags", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
},
}
err := os.FlagSet().Parse([]string{"--some-unknown", "foo"})
require.Error(t, err)
})
t.Run("RegexValid", func(t *testing.T) {
t.Parallel()
var regexpString clibase.Regexp
os := clibase.OptionSet{
clibase.Option{
Name: "RegexpString",
Value: &regexpString,
Flag: "regexp-string",
},
}
err := os.FlagSet().Parse([]string{"--regexp-string", "$test^"})
require.NoError(t, err)
})
t.Run("RegexInvalid", func(t *testing.T) {
t.Parallel()
var regexpString clibase.Regexp
os := clibase.OptionSet{
clibase.Option{
Name: "RegexpString",
Value: &regexpString,
Flag: "regexp-string",
},
}
err := os.FlagSet().Parse([]string{"--regexp-string", "(("})
require.Error(t, err)
})
}
func TestOptionSet_ParseEnv(t *testing.T) {
t.Parallel()
t.Run("SimpleString", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Env: "WORKSPACE_NAME",
},
}
err := os.ParseEnv([]clibase.EnvVar{
{Name: "WORKSPACE_NAME", Value: "foo"},
})
require.NoError(t, err)
require.EqualValues(t, "foo", workspaceName)
})
t.Run("EmptyValue", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Default: "defname",
Env: "WORKSPACE_NAME",
},
}
err := os.SetDefaults()
require.NoError(t, err)
err = os.ParseEnv(clibase.ParseEnviron([]string{"CODER_WORKSPACE_NAME="}, "CODER_"))
require.NoError(t, err)
require.EqualValues(t, "defname", workspaceName)
})
t.Run("StringSlice", func(t *testing.T) {
t.Parallel()
var actual clibase.StringArray
expected := []string{"foo", "bar", "baz"}
os := clibase.OptionSet{
clibase.Option{
Name: "name",
Value: &actual,
Env: "NAMES",
},
}
err := os.SetDefaults()
require.NoError(t, err)
err = os.ParseEnv([]clibase.EnvVar{
{Name: "NAMES", Value: "foo,bar,baz"},
})
require.NoError(t, err)
require.EqualValues(t, expected, actual)
})
t.Run("StructMapStringString", func(t *testing.T) {
t.Parallel()
var actual clibase.Struct[map[string]string]
expected := map[string]string{"foo": "bar", "baz": "zap"}
os := clibase.OptionSet{
clibase.Option{
Name: "labels",
Value: &actual,
Env: "LABELS",
},
}
err := os.SetDefaults()
require.NoError(t, err)
err = os.ParseEnv([]clibase.EnvVar{
{Name: "LABELS", Value: `{"foo":"bar","baz":"zap"}`},
})
require.NoError(t, err)
require.EqualValues(t, expected, actual.Value)
})
t.Run("Homebrew", func(t *testing.T) {
t.Parallel()
var agentToken clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Agent Token",
Value: &agentToken,
Env: "AGENT_TOKEN",
},
}
err := os.ParseEnv([]clibase.EnvVar{
{Name: "HOMEBREW_AGENT_TOKEN", Value: "foo"},
})
require.NoError(t, err)
require.EqualValues(t, "foo", agentToken)
})
}
func TestOptionSet_JsonMarshal(t *testing.T) {
t.Parallel()
// This unit test ensures if the source optionset is missing the option
// and cannot determine the type, it will not panic. The unmarshal will
// succeed with a best effort.
t.Run("MissingSrcOption", func(t *testing.T) {
t.Parallel()
var str clibase.String = "something"
var arr clibase.StringArray = []string{"foo", "bar"}
opts := clibase.OptionSet{
clibase.Option{
Name: "StringOpt",
Value: &str,
},
clibase.Option{
Name: "ArrayOpt",
Value: &arr,
},
}
data, err := json.Marshal(opts)
require.NoError(t, err, "marshal option set")
tgt := clibase.OptionSet{}
err = json.Unmarshal(data, &tgt)
require.NoError(t, err, "unmarshal option set")
for i := range opts {
compareOptionsExceptValues(t, opts[i], tgt[i])
require.Empty(t, tgt[i].Value.String(), "unknown value types are empty")
}
})
t.Run("RegexCase", func(t *testing.T) {
t.Parallel()
val := clibase.Regexp(*regexp.MustCompile(".*"))
opts := clibase.OptionSet{
clibase.Option{
Name: "Regex",
Value: &val,
Default: ".*",
},
}
data, err := json.Marshal(opts)
require.NoError(t, err, "marshal option set")
var foundVal clibase.Regexp
newOpts := clibase.OptionSet{
clibase.Option{
Name: "Regex",
Value: &foundVal,
},
}
err = json.Unmarshal(data, &newOpts)
require.NoError(t, err, "unmarshal option set")
require.EqualValues(t, opts[0].Value.String(), newOpts[0].Value.String())
})
t.Run("AllValues", func(t *testing.T) {
t.Parallel()
vals := coderdtest.DeploymentValues(t)
opts := vals.Options()
sources := []clibase.ValueSource{
clibase.ValueSourceNone,
clibase.ValueSourceFlag,
clibase.ValueSourceEnv,
clibase.ValueSourceYAML,
clibase.ValueSourceDefault,
}
for i := range opts {
opts[i].ValueSource = sources[i%len(sources)]
}
data, err := json.Marshal(opts)
require.NoError(t, err, "marshal option set")
newOpts := (&codersdk.DeploymentValues{}).Options()
err = json.Unmarshal(data, &newOpts)
require.NoError(t, err, "unmarshal option set")
for i := range opts {
exp := opts[i]
found := newOpts[i]
compareOptionsExceptValues(t, exp, found)
compareValues(t, exp, found)
}
thirdOpts := (&codersdk.DeploymentValues{}).Options()
data, err = json.Marshal(newOpts)
require.NoError(t, err, "marshal option set")
err = json.Unmarshal(data, &thirdOpts)
require.NoError(t, err, "unmarshal option set")
// Compare to the original opts again
for i := range opts {
exp := opts[i]
found := thirdOpts[i]
compareOptionsExceptValues(t, exp, found)
compareValues(t, exp, found)
}
})
}
func compareOptionsExceptValues(t *testing.T, exp, found clibase.Option) {
t.Helper()
require.Equalf(t, exp.Name, found.Name, "option name %q", exp.Name)
require.Equalf(t, exp.Description, found.Description, "option description %q", exp.Name)
require.Equalf(t, exp.Required, found.Required, "option required %q", exp.Name)
require.Equalf(t, exp.Flag, found.Flag, "option flag %q", exp.Name)
require.Equalf(t, exp.FlagShorthand, found.FlagShorthand, "option flag shorthand %q", exp.Name)
require.Equalf(t, exp.Env, found.Env, "option env %q", exp.Name)
require.Equalf(t, exp.YAML, found.YAML, "option yaml %q", exp.Name)
require.Equalf(t, exp.Default, found.Default, "option default %q", exp.Name)
require.Equalf(t, exp.ValueSource, found.ValueSource, "option value source %q", exp.Name)
require.Equalf(t, exp.Hidden, found.Hidden, "option hidden %q", exp.Name)
require.Equalf(t, exp.Annotations, found.Annotations, "option annotations %q", exp.Name)
require.Equalf(t, exp.Group, found.Group, "option group %q", exp.Name)
// UseInstead is the same comparison problem, just check the length
require.Equalf(t, len(exp.UseInstead), len(found.UseInstead), "option use instead %q", exp.Name)
}
func compareValues(t *testing.T, exp, found clibase.Option) {
t.Helper()
if (exp.Value == nil || found.Value == nil) || (exp.Value.String() != found.Value.String() && found.Value.String() == "") {
// If the string values are different, this can be a "nil" issue.
// So only run this case if the found string is the empty string.
// We use MarshalYAML for struct strings, and it will return an
// empty string '""' for nil slices/maps/etc.
// So use json to compare.
expJSON, err := json.Marshal(exp.Value)
require.NoError(t, err, "marshal")
foundJSON, err := json.Marshal(found.Value)
require.NoError(t, err, "marshal")
expJSON = normalizeJSON(expJSON)
foundJSON = normalizeJSON(foundJSON)
assert.Equalf(t, string(expJSON), string(foundJSON), "option value %q", exp.Name)
} else {
assert.Equal(t,
exp.Value.String(),
found.Value.String(),
"option value %q", exp.Name)
}
}
// normalizeJSON handles the fact that an empty map/slice is not the same
// as a nil empty/slice. For our purposes, they are the same.
func normalizeJSON(data []byte) []byte {
if bytes.Equal(data, []byte("[]")) || bytes.Equal(data, []byte("{}")) {
return []byte("null")
}
return data
}
-567
View File
@@ -1,567 +0,0 @@
package clibase
import (
"encoding/csv"
"encoding/json"
"fmt"
"net"
"net/url"
"reflect"
"regexp"
"strconv"
"strings"
"time"
"github.com/spf13/pflag"
"golang.org/x/xerrors"
"gopkg.in/yaml.v3"
)
// NoOptDefValuer describes behavior when no
// option is passed into the flag.
//
// This is useful for boolean or otherwise binary flags.
type NoOptDefValuer interface {
NoOptDefValue() string
}
// Validator is a wrapper around a pflag.Value that allows for validation
// of the value after or before it has been set.
type Validator[T pflag.Value] struct {
Value T
// validate is called after the value is set.
validate func(T) error
}
func Validate[T pflag.Value](opt T, validate func(value T) error) *Validator[T] {
return &Validator[T]{Value: opt, validate: validate}
}
func (i *Validator[T]) String() string {
return i.Value.String()
}
func (i *Validator[T]) Set(input string) error {
err := i.Value.Set(input)
if err != nil {
return err
}
if i.validate != nil {
err = i.validate(i.Value)
if err != nil {
return err
}
}
return nil
}
func (i *Validator[T]) Type() string {
return i.Value.Type()
}
// values.go contains a standard set of value types that can be used as
// Option Values.
type Int64 int64
func Int64Of(i *int64) *Int64 {
return (*Int64)(i)
}
func (i *Int64) Set(s string) error {
ii, err := strconv.ParseInt(s, 10, 64)
*i = Int64(ii)
return err
}
func (i Int64) Value() int64 {
return int64(i)
}
func (i Int64) String() string {
return strconv.Itoa(int(i))
}
func (Int64) Type() string {
return "int"
}
type Bool bool
func BoolOf(b *bool) *Bool {
return (*Bool)(b)
}
func (b *Bool) Set(s string) error {
if s == "" {
*b = Bool(false)
return nil
}
bb, err := strconv.ParseBool(s)
*b = Bool(bb)
return err
}
func (*Bool) NoOptDefValue() string {
return "true"
}
func (b Bool) String() string {
return strconv.FormatBool(bool(b))
}
func (b Bool) Value() bool {
return bool(b)
}
func (Bool) Type() string {
return "bool"
}
type String string
func StringOf(s *string) *String {
return (*String)(s)
}
func (*String) NoOptDefValue() string {
return ""
}
func (s *String) Set(v string) error {
*s = String(v)
return nil
}
func (s String) String() string {
return string(s)
}
func (s String) Value() string {
return string(s)
}
func (String) Type() string {
return "string"
}
var _ pflag.SliceValue = &StringArray{}
// StringArray is a slice of strings that implements pflag.Value and pflag.SliceValue.
type StringArray []string
func StringArrayOf(ss *[]string) *StringArray {
return (*StringArray)(ss)
}
func (s *StringArray) Append(v string) error {
*s = append(*s, v)
return nil
}
func (s *StringArray) Replace(vals []string) error {
*s = vals
return nil
}
func (s *StringArray) GetSlice() []string {
return *s
}
func readAsCSV(v string) ([]string, error) {
return csv.NewReader(strings.NewReader(v)).Read()
}
func writeAsCSV(vals []string) string {
var sb strings.Builder
err := csv.NewWriter(&sb).Write(vals)
if err != nil {
return fmt.Sprintf("error: %s", err)
}
return sb.String()
}
func (s *StringArray) Set(v string) error {
if v == "" {
*s = nil
return nil
}
ss, err := readAsCSV(v)
if err != nil {
return err
}
*s = append(*s, ss...)
return nil
}
func (s StringArray) String() string {
return writeAsCSV([]string(s))
}
func (s StringArray) Value() []string {
return []string(s)
}
func (StringArray) Type() string {
return "string-array"
}
type Duration time.Duration
func DurationOf(d *time.Duration) *Duration {
return (*Duration)(d)
}
func (d *Duration) Set(v string) error {
dd, err := time.ParseDuration(v)
*d = Duration(dd)
return err
}
func (d *Duration) Value() time.Duration {
return time.Duration(*d)
}
func (d *Duration) String() string {
return time.Duration(*d).String()
}
func (Duration) Type() string {
return "duration"
}
func (d *Duration) MarshalYAML() (interface{}, error) {
return yaml.Node{
Kind: yaml.ScalarNode,
Value: d.String(),
}, nil
}
func (d *Duration) UnmarshalYAML(n *yaml.Node) error {
return d.Set(n.Value)
}
type URL url.URL
func URLOf(u *url.URL) *URL {
return (*URL)(u)
}
func (u *URL) Set(v string) error {
uu, err := url.Parse(v)
if err != nil {
return err
}
*u = URL(*uu)
return nil
}
func (u *URL) String() string {
uu := url.URL(*u)
return uu.String()
}
func (u *URL) MarshalYAML() (interface{}, error) {
return yaml.Node{
Kind: yaml.ScalarNode,
Value: u.String(),
}, nil
}
func (u *URL) UnmarshalYAML(n *yaml.Node) error {
return u.Set(n.Value)
}
func (u *URL) MarshalJSON() ([]byte, error) {
return json.Marshal(u.String())
}
func (u *URL) UnmarshalJSON(b []byte) error {
var s string
err := json.Unmarshal(b, &s)
if err != nil {
return err
}
return u.Set(s)
}
func (*URL) Type() string {
return "url"
}
func (u *URL) Value() *url.URL {
return (*url.URL)(u)
}
// HostPort is a host:port pair.
type HostPort struct {
Host string
Port string
}
func (hp *HostPort) Set(v string) error {
if v == "" {
return xerrors.Errorf("must not be empty")
}
var err error
hp.Host, hp.Port, err = net.SplitHostPort(v)
return err
}
func (hp *HostPort) String() string {
if hp.Host == "" && hp.Port == "" {
return ""
}
// Warning: net.JoinHostPort must be used over concatenation to support
// IPv6 addresses.
return net.JoinHostPort(hp.Host, hp.Port)
}
func (hp *HostPort) MarshalJSON() ([]byte, error) {
return json.Marshal(hp.String())
}
func (hp *HostPort) UnmarshalJSON(b []byte) error {
var s string
err := json.Unmarshal(b, &s)
if err != nil {
return err
}
if s == "" {
hp.Host = ""
hp.Port = ""
return nil
}
return hp.Set(s)
}
func (hp *HostPort) MarshalYAML() (interface{}, error) {
return yaml.Node{
Kind: yaml.ScalarNode,
Value: hp.String(),
}, nil
}
func (hp *HostPort) UnmarshalYAML(n *yaml.Node) error {
return hp.Set(n.Value)
}
func (*HostPort) Type() string {
return "host:port"
}
var (
_ yaml.Marshaler = new(Struct[struct{}])
_ yaml.Unmarshaler = new(Struct[struct{}])
)
// Struct is a special value type that encodes an arbitrary struct.
// It implements the flag.Value interface, but in general these values should
// only be accepted via config for ergonomics.
//
// The string encoding type is YAML.
type Struct[T any] struct {
Value T
}
//nolint:revive
func (s *Struct[T]) Set(v string) error {
return yaml.Unmarshal([]byte(v), &s.Value)
}
//nolint:revive
func (s *Struct[T]) String() string {
byt, err := yaml.Marshal(s.Value)
if err != nil {
return "decode failed: " + err.Error()
}
return string(byt)
}
func (s *Struct[T]) MarshalYAML() (interface{}, error) {
var n yaml.Node
err := n.Encode(s.Value)
if err != nil {
return nil, err
}
return n, nil
}
func (s *Struct[T]) UnmarshalYAML(n *yaml.Node) error {
// HACK: for compatibility with flags, we use nil slices instead of empty
// slices. In most cases, nil slices and empty slices are treated
// the same, so this behavior may be removed at some point.
if typ := reflect.TypeOf(s.Value); typ.Kind() == reflect.Slice && len(n.Content) == 0 {
reflect.ValueOf(&s.Value).Elem().Set(reflect.Zero(typ))
return nil
}
return n.Decode(&s.Value)
}
//nolint:revive
func (s *Struct[T]) Type() string {
return fmt.Sprintf("struct[%T]", s.Value)
}
func (s *Struct[T]) MarshalJSON() ([]byte, error) {
return json.Marshal(s.Value)
}
func (s *Struct[T]) UnmarshalJSON(b []byte) error {
return json.Unmarshal(b, &s.Value)
}
// DiscardValue does nothing but implements the pflag.Value interface.
// It's useful in cases where you want to accept an option, but access the
// underlying value directly instead of through the Option methods.
var DiscardValue discardValue
type discardValue struct{}
func (discardValue) Set(string) error {
return nil
}
func (discardValue) String() string {
return ""
}
func (discardValue) Type() string {
return "discard"
}
func (discardValue) UnmarshalJSON([]byte) error {
return nil
}
// jsonValue is intentionally not exported. It is just used to store the raw JSON
// data for a value to defer it's unmarshal. It implements the pflag.Value to be
// usable in an Option.
type jsonValue json.RawMessage
func (jsonValue) Set(string) error {
return xerrors.Errorf("json value is read-only")
}
func (jsonValue) String() string {
return ""
}
func (jsonValue) Type() string {
return "json"
}
func (j *jsonValue) UnmarshalJSON(data []byte) error {
if j == nil {
return xerrors.New("json.RawMessage: UnmarshalJSON on nil pointer")
}
*j = append((*j)[0:0], data...)
return nil
}
var _ pflag.Value = (*Enum)(nil)
type Enum struct {
Choices []string
Value *string
}
func EnumOf(v *string, choices ...string) *Enum {
return &Enum{
Choices: choices,
Value: v,
}
}
func (e *Enum) Set(v string) error {
for _, c := range e.Choices {
if v == c {
*e.Value = v
return nil
}
}
return xerrors.Errorf("invalid choice: %s, should be one of %v", v, e.Choices)
}
func (e *Enum) Type() string {
return fmt.Sprintf("enum[%v]", strings.Join(e.Choices, "|"))
}
func (e *Enum) String() string {
return *e.Value
}
type Regexp regexp.Regexp
func (r *Regexp) MarshalJSON() ([]byte, error) {
return json.Marshal(r.String())
}
func (r *Regexp) UnmarshalJSON(data []byte) error {
var source string
err := json.Unmarshal(data, &source)
if err != nil {
return err
}
exp, err := regexp.Compile(source)
if err != nil {
return xerrors.Errorf("invalid regex expression: %w", err)
}
*r = Regexp(*exp)
return nil
}
func (r *Regexp) MarshalYAML() (interface{}, error) {
return yaml.Node{
Kind: yaml.ScalarNode,
Value: r.String(),
}, nil
}
func (r *Regexp) UnmarshalYAML(n *yaml.Node) error {
return r.Set(n.Value)
}
func (r *Regexp) Set(v string) error {
exp, err := regexp.Compile(v)
if err != nil {
return xerrors.Errorf("invalid regex expression: %w", err)
}
*r = Regexp(*exp)
return nil
}
func (r Regexp) String() string {
return r.Value().String()
}
func (r *Regexp) Value() *regexp.Regexp {
if r == nil {
return nil
}
return (*regexp.Regexp)(r)
}
func (Regexp) Type() string {
return "regexp"
}
var _ pflag.Value = (*YAMLConfigPath)(nil)
// YAMLConfigPath is a special value type that encodes a path to a YAML
// configuration file where options are read from.
type YAMLConfigPath string
func (p *YAMLConfigPath) Set(v string) error {
*p = YAMLConfigPath(v)
return nil
}
func (p *YAMLConfigPath) String() string {
return string(*p)
}
func (*YAMLConfigPath) Type() string {
return "yaml-config-path"
}
-295
View File
@@ -1,295 +0,0 @@
package clibase
import (
"errors"
"fmt"
"strings"
"github.com/mitchellh/go-wordwrap"
"golang.org/x/xerrors"
"gopkg.in/yaml.v3"
)
var (
_ yaml.Marshaler = new(OptionSet)
_ yaml.Unmarshaler = new(OptionSet)
)
// deepMapNode returns the mapping node at the given path,
// creating it if it doesn't exist.
func deepMapNode(n *yaml.Node, path []string, headComment string) *yaml.Node {
if len(path) == 0 {
return n
}
// Name is every two nodes.
for i := 0; i < len(n.Content)-1; i += 2 {
if n.Content[i].Value == path[0] {
// Found matching name, recurse.
return deepMapNode(n.Content[i+1], path[1:], headComment)
}
}
// Not found, create it.
nameNode := yaml.Node{
Kind: yaml.ScalarNode,
Value: path[0],
HeadComment: headComment,
}
valueNode := yaml.Node{
Kind: yaml.MappingNode,
}
n.Content = append(n.Content, &nameNode)
n.Content = append(n.Content, &valueNode)
return deepMapNode(&valueNode, path[1:], headComment)
}
// MarshalYAML converts the option set to a YAML node, that can be
// converted into bytes via yaml.Marshal.
//
// The node is returned to enable post-processing higher up in
// the stack.
//
// It is isomorphic with FromYAML.
func (optSet *OptionSet) MarshalYAML() (any, error) {
root := yaml.Node{
Kind: yaml.MappingNode,
}
for _, opt := range *optSet {
if opt.YAML == "" {
continue
}
defValue := opt.Default
if defValue == "" {
defValue = "<unset>"
}
comment := wordwrap.WrapString(
fmt.Sprintf("%s\n(default: %s, type: %s)", opt.Description, defValue, opt.Value.Type()),
80,
)
nameNode := yaml.Node{
Kind: yaml.ScalarNode,
Value: opt.YAML,
HeadComment: comment,
}
var valueNode yaml.Node
if opt.Value == nil {
valueNode = yaml.Node{
Kind: yaml.ScalarNode,
Value: "null",
}
} else if m, ok := opt.Value.(yaml.Marshaler); ok {
v, err := m.MarshalYAML()
if err != nil {
return nil, xerrors.Errorf(
"marshal %q: %w", opt.Name, err,
)
}
valueNode, ok = v.(yaml.Node)
if !ok {
return nil, xerrors.Errorf(
"marshal %q: unexpected underlying type %T",
opt.Name, v,
)
}
} else {
// The all-other types case.
//
// A bit of a hack, we marshal and then unmarshal to get
// the underlying node.
byt, err := yaml.Marshal(opt.Value)
if err != nil {
return nil, xerrors.Errorf(
"marshal %q: %w", opt.Name, err,
)
}
var docNode yaml.Node
err = yaml.Unmarshal(byt, &docNode)
if err != nil {
return nil, xerrors.Errorf(
"unmarshal %q: %w", opt.Name, err,
)
}
if len(docNode.Content) != 1 {
return nil, xerrors.Errorf(
"unmarshal %q: expected one node, got %d",
opt.Name, len(docNode.Content),
)
}
valueNode = *docNode.Content[0]
}
var group []string
for _, g := range opt.Group.Ancestry() {
if g.YAML == "" {
return nil, xerrors.Errorf(
"group yaml name is empty for %q, groups: %+v",
opt.Name,
opt.Group,
)
}
group = append(group, g.YAML)
}
var groupDesc string
if opt.Group != nil {
groupDesc = wordwrap.WrapString(opt.Group.Description, 80)
}
parentValueNode := deepMapNode(
&root, group,
groupDesc,
)
parentValueNode.Content = append(
parentValueNode.Content,
&nameNode,
&valueNode,
)
}
return &root, nil
}
// mapYAMLNodes converts parent into a map with keys of form "group.subgroup.option"
// and values as the corresponding YAML nodes.
func mapYAMLNodes(parent *yaml.Node) (map[string]*yaml.Node, error) {
if parent.Kind != yaml.MappingNode {
return nil, xerrors.Errorf("expected mapping node, got type %v", parent.Kind)
}
if len(parent.Content)%2 != 0 {
return nil, xerrors.Errorf("expected an even number of k/v pairs, got %d", len(parent.Content))
}
var (
key string
m = make(map[string]*yaml.Node, len(parent.Content)/2)
merr error
)
for i, child := range parent.Content {
if i%2 == 0 {
if child.Kind != yaml.ScalarNode {
// We immediately because the rest of the code is bound to fail
// if we don't know to expect a key or a value.
return nil, xerrors.Errorf("expected scalar node for key, got type %v", child.Kind)
}
key = child.Value
continue
}
// We don't know if this is a grouped simple option or complex option,
// so we store both "key" and "group.key". Since we're storing pointers,
// the additional memory is of little concern.
m[key] = child
if child.Kind != yaml.MappingNode {
continue
}
sub, err := mapYAMLNodes(child)
if err != nil {
merr = errors.Join(merr, xerrors.Errorf("mapping node %q: %w", key, err))
continue
}
for k, v := range sub {
m[key+"."+k] = v
}
}
return m, nil
}
func (o *Option) setFromYAMLNode(n *yaml.Node) error {
o.ValueSource = ValueSourceYAML
if um, ok := o.Value.(yaml.Unmarshaler); ok {
return um.UnmarshalYAML(n)
}
switch n.Kind {
case yaml.ScalarNode:
return o.Value.Set(n.Value)
case yaml.SequenceNode:
// We treat empty values as nil for consistency with other option
// mechanisms.
if len(n.Content) == 0 {
o.Value = nil
return nil
}
return n.Decode(o.Value)
case yaml.MappingNode:
return xerrors.Errorf("mapping nodes must implement yaml.Unmarshaler")
default:
return xerrors.Errorf("unexpected node kind %v", n.Kind)
}
}
// UnmarshalYAML converts the given YAML node into the option set.
// It is isomorphic with ToYAML.
func (optSet *OptionSet) UnmarshalYAML(rootNode *yaml.Node) error {
// The rootNode will be a DocumentNode if it's read from a file. We do
// not support multiple documents in a single file.
if rootNode.Kind == yaml.DocumentNode {
if len(rootNode.Content) != 1 {
return xerrors.Errorf("expected one node in document, got %d", len(rootNode.Content))
}
rootNode = rootNode.Content[0]
}
yamlNodes, err := mapYAMLNodes(rootNode)
if err != nil {
return xerrors.Errorf("mapping nodes: %w", err)
}
matchedNodes := make(map[string]*yaml.Node, len(yamlNodes))
var merr error
for i := range *optSet {
opt := &(*optSet)[i]
if opt.YAML == "" {
continue
}
var group []string
for _, g := range opt.Group.Ancestry() {
if g.YAML == "" {
return xerrors.Errorf(
"group yaml name is empty for %q, groups: %+v",
opt.Name,
opt.Group,
)
}
group = append(group, g.YAML)
delete(yamlNodes, strings.Join(group, "."))
}
key := strings.Join(append(group, opt.YAML), ".")
node, ok := yamlNodes[key]
if !ok {
continue
}
matchedNodes[key] = node
if opt.ValueSource != ValueSourceNone {
continue
}
if err := opt.setFromYAMLNode(node); err != nil {
merr = errors.Join(merr, xerrors.Errorf("setting %q: %w", opt.YAML, err))
}
}
// Remove all matched nodes and their descendants from yamlNodes so we
// can accurately report unknown options.
for k := range yamlNodes {
var key string
for _, part := range strings.Split(k, ".") {
if key != "" {
key += "."
}
key += part
if _, ok := matchedNodes[key]; ok {
delete(yamlNodes, k)
}
}
}
for k := range yamlNodes {
merr = errors.Join(merr, xerrors.Errorf("unknown option %q", k))
}
return merr
}
-202
View File
@@ -1,202 +0,0 @@
package clibase_test
import (
"testing"
"github.com/spf13/pflag"
"github.com/stretchr/testify/require"
"golang.org/x/exp/slices"
"gopkg.in/yaml.v3"
"github.com/coder/coder/v2/cli/clibase"
)
func TestOptionSet_YAML(t *testing.T) {
t.Parallel()
t.Run("RequireKey", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Default: "billie",
},
}
node, err := os.MarshalYAML()
require.NoError(t, err)
require.Len(t, node.(*yaml.Node).Content, 0)
})
t.Run("SimpleString", func(t *testing.T) {
t.Parallel()
var workspaceName clibase.String
os := clibase.OptionSet{
clibase.Option{
Name: "Workspace Name",
Value: &workspaceName,
Default: "billie",
Description: "The workspace's name.",
Group: &clibase.Group{YAML: "names"},
YAML: "workspaceName",
},
}
err := os.SetDefaults()
require.NoError(t, err)
n, err := os.MarshalYAML()
require.NoError(t, err)
// Visually inspect for now.
byt, err := yaml.Marshal(n)
require.NoError(t, err)
t.Logf("Raw YAML:\n%s", string(byt))
})
}
func TestOptionSet_YAMLUnknownOptions(t *testing.T) {
t.Parallel()
os := clibase.OptionSet{
{
Name: "Workspace Name",
Default: "billie",
Description: "The workspace's name.",
YAML: "workspaceName",
Value: new(clibase.String),
},
}
const yamlDoc = `something: else`
err := yaml.Unmarshal([]byte(yamlDoc), &os)
require.Error(t, err)
require.Empty(t, os[0].Value.String())
os[0].YAML = "something"
err = yaml.Unmarshal([]byte(yamlDoc), &os)
require.NoError(t, err)
require.Equal(t, "else", os[0].Value.String())
}
// TestOptionSet_YAMLIsomorphism tests that the YAML representations of an
// OptionSet converts to the same OptionSet when read back in.
func TestOptionSet_YAMLIsomorphism(t *testing.T) {
t.Parallel()
// This is used to form a generic.
//nolint:unused
type kid struct {
Name string `yaml:"name"`
Age int `yaml:"age"`
}
for _, tc := range []struct {
name string
os clibase.OptionSet
zeroValue func() pflag.Value
}{
{
name: "SimpleString",
os: clibase.OptionSet{
{
Name: "Workspace Name",
Default: "billie",
Description: "The workspace's name.",
Group: &clibase.Group{YAML: "names"},
YAML: "workspaceName",
},
},
zeroValue: func() pflag.Value {
return clibase.StringOf(new(string))
},
},
{
name: "Array",
os: clibase.OptionSet{
{
YAML: "names",
Default: "jill,jack,joan",
},
},
zeroValue: func() pflag.Value {
return clibase.StringArrayOf(&[]string{})
},
},
{
name: "ComplexObject",
os: clibase.OptionSet{
{
YAML: "kids",
Default: `- name: jill
age: 12
- name: jack
age: 13`,
},
},
zeroValue: func() pflag.Value {
return &clibase.Struct[[]kid]{}
},
},
{
name: "DeepGroup",
os: clibase.OptionSet{
{
YAML: "names",
Default: "jill,jack,joan",
Group: &clibase.Group{YAML: "kids", Parent: &clibase.Group{YAML: "family"}},
},
},
zeroValue: func() pflag.Value {
return clibase.StringArrayOf(&[]string{})
},
},
} {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
// Set initial values.
for i := range tc.os {
tc.os[i].Value = tc.zeroValue()
}
err := tc.os.SetDefaults()
require.NoError(t, err)
y, err := tc.os.MarshalYAML()
require.NoError(t, err)
toByt, err := yaml.Marshal(y)
require.NoError(t, err)
t.Logf("Raw YAML:\n%s", string(toByt))
var y2 yaml.Node
err = yaml.Unmarshal(toByt, &y2)
require.NoError(t, err)
os2 := slices.Clone(tc.os)
for i := range os2 {
os2[i].Value = tc.zeroValue()
os2[i].ValueSource = clibase.ValueSourceNone
}
// os2 values should be zeroed whereas tc.os should be
// set to defaults.
// This check makes sure we aren't mixing pointers.
require.NotEqual(t, tc.os, os2)
err = os2.UnmarshalYAML(&y2)
require.NoError(t, err)
want := tc.os
for i := range want {
want[i].ValueSource = clibase.ValueSourceYAML
}
require.Equal(t, tc.os, os2)
})
}
}
+211
View File
@@ -0,0 +1,211 @@
package clilog
import (
"context"
"fmt"
"io"
"os"
"regexp"
"strings"
"golang.org/x/xerrors"
"cdr.dev/slog"
"cdr.dev/slog/sloggers/sloghuman"
"cdr.dev/slog/sloggers/slogjson"
"cdr.dev/slog/sloggers/slogstackdriver"
"github.com/coder/coder/v2/coderd/tracing"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
type (
Option func(*Builder)
Builder struct {
Filter []string
Human string
JSON string
Stackdriver string
Trace bool
Verbose bool
}
)
func New(opts ...Option) *Builder {
b := &Builder{}
for _, opt := range opts {
opt(b)
}
return b
}
func WithFilter(filters ...string) Option {
return func(b *Builder) {
b.Filter = filters
}
}
func WithHuman(loc string) Option {
return func(b *Builder) {
b.Human = loc
}
}
func WithJSON(loc string) Option {
return func(b *Builder) {
b.JSON = loc
}
}
func WithStackdriver(loc string) Option {
return func(b *Builder) {
b.Stackdriver = loc
}
}
func WithTrace() Option {
return func(b *Builder) {
b.Trace = true
}
}
func WithVerbose() Option {
return func(b *Builder) {
b.Verbose = true
}
}
func FromDeploymentValues(vals *codersdk.DeploymentValues) Option {
return func(b *Builder) {
b.Filter = vals.Logging.Filter.Value()
b.Human = vals.Logging.Human.Value()
b.JSON = vals.Logging.JSON.Value()
b.Stackdriver = vals.Logging.Stackdriver.Value()
b.Trace = vals.Trace.Enable.Value()
b.Verbose = vals.Verbose.Value()
}
}
func (b *Builder) Build(inv *serpent.Invocation) (log slog.Logger, closeLog func(), err error) {
var (
sinks = []slog.Sink{}
closers = []func() error{}
)
defer func() {
if err != nil {
for _, closer := range closers {
_ = closer()
}
}
}()
noopClose := func() {}
addSinkIfProvided := func(sinkFn func(io.Writer) slog.Sink, loc string) error {
switch loc {
case "":
case "/dev/stdout":
sinks = append(sinks, sinkFn(inv.Stdout))
case "/dev/stderr":
sinks = append(sinks, sinkFn(inv.Stderr))
default:
fi, err := os.OpenFile(loc, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0o644)
if err != nil {
return xerrors.Errorf("open log file %q: %w", loc, err)
}
closers = append(closers, fi.Close)
sinks = append(sinks, sinkFn(fi))
}
return nil
}
err = addSinkIfProvided(sloghuman.Sink, b.Human)
if err != nil {
return slog.Logger{}, noopClose, xerrors.Errorf("add human sink: %w", err)
}
err = addSinkIfProvided(slogjson.Sink, b.JSON)
if err != nil {
return slog.Logger{}, noopClose, xerrors.Errorf("add json sink: %w", err)
}
err = addSinkIfProvided(slogstackdriver.Sink, b.Stackdriver)
if err != nil {
return slog.Logger{}, noopClose, xerrors.Errorf("add stackdriver sink: %w", err)
}
if b.Trace {
sinks = append(sinks, tracing.SlogSink{})
}
// User should log to null device if they don't want logs.
if len(sinks) == 0 {
return slog.Logger{}, noopClose, xerrors.New("no loggers provided, use /dev/null to disable logging")
}
filter := &debugFilterSink{next: sinks}
err = filter.compile(b.Filter)
if err != nil {
return slog.Logger{}, noopClose, xerrors.Errorf("compile filters: %w", err)
}
level := slog.LevelInfo
// Debug logging is always enabled if a filter is present.
if b.Verbose || filter.re != nil {
level = slog.LevelDebug
}
return inv.Logger.AppendSinks(filter).Leveled(level), func() {
for _, closer := range closers {
_ = closer()
}
}, nil
}
var _ slog.Sink = &debugFilterSink{}
type debugFilterSink struct {
next []slog.Sink
re *regexp.Regexp
}
func (f *debugFilterSink) compile(res []string) error {
if len(res) == 0 {
return nil
}
var reb strings.Builder
for i, re := range res {
_, _ = fmt.Fprintf(&reb, "(%s)", re)
if i != len(res)-1 {
_, _ = reb.WriteRune('|')
}
}
re, err := regexp.Compile(reb.String())
if err != nil {
return xerrors.Errorf("compile regex: %w", err)
}
f.re = re
return nil
}
func (f *debugFilterSink) LogEntry(ctx context.Context, ent slog.SinkEntry) {
if ent.Level == slog.LevelDebug {
logName := strings.Join(ent.LoggerNames, ".")
if f.re != nil && !f.re.MatchString(logName) && !f.re.MatchString(ent.Message) {
return
}
}
for _, sink := range f.next {
sink.LogEntry(ctx, ent)
}
}
func (f *debugFilterSink) Sync() {
for _, sink := range f.next {
sink.Sync()
}
}
+243
View File
@@ -0,0 +1,243 @@
package clilog_test
import (
"encoding/json"
"io/fs"
"os"
"path/filepath"
"strings"
"testing"
"github.com/coder/coder/v2/cli/clilog"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestBuilder(t *testing.T) {
t.Parallel()
t.Run("NoConfiguration", func(t *testing.T) {
t.Parallel()
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t),
}
err := cmd.Invoke().Run()
require.ErrorContains(t, err, "no loggers provided, use /dev/null to disable logging")
})
t.Run("Verbose", func(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "test.log")
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t,
clilog.WithHuman(tempFile),
clilog.WithVerbose(),
),
}
err := cmd.Invoke().Run()
require.NoError(t, err)
assertLogs(t, tempFile, debugLog, infoLog, warnLog, filterLog)
})
t.Run("WithFilter", func(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "test.log")
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t,
clilog.WithHuman(tempFile),
// clilog.WithVerbose(), // implicit
clilog.WithFilter("important debug message"),
),
}
err := cmd.Invoke().Run()
require.NoError(t, err)
assertLogs(t, tempFile, infoLog, warnLog, filterLog)
})
t.Run("WithHuman", func(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "test.log")
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t, clilog.WithHuman(tempFile)),
}
err := cmd.Invoke().Run()
require.NoError(t, err)
assertLogs(t, tempFile, infoLog, warnLog)
})
t.Run("WithJSON", func(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "test.log")
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t, clilog.WithJSON(tempFile), clilog.WithVerbose()),
}
err := cmd.Invoke().Run()
require.NoError(t, err)
assertLogsJSON(t, tempFile, debug, debugLog, info, infoLog, warn, warnLog, debug, filterLog)
})
t.Run("FromDeploymentValues", func(t *testing.T) {
t.Parallel()
t.Run("Defaults", func(t *testing.T) {
stdoutPath := filepath.Join(t.TempDir(), "stdout")
stderrPath := filepath.Join(t.TempDir(), "stderr")
stdout, err := os.OpenFile(stdoutPath, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0o644)
require.NoError(t, err)
t.Cleanup(func() { _ = stdout.Close() })
stderr, err := os.OpenFile(stderrPath, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0o644)
require.NoError(t, err)
t.Cleanup(func() { _ = stderr.Close() })
// Use the default deployment values.
dv := coderdtest.DeploymentValues(t)
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t, clilog.FromDeploymentValues(dv)),
}
inv := cmd.Invoke()
inv.Stdout = stdout
inv.Stderr = stderr
err = inv.Run()
require.NoError(t, err)
assertLogs(t, stdoutPath, "")
assertLogs(t, stderrPath, infoLog, warnLog)
})
t.Run("Override", func(t *testing.T) {
tempFile := filepath.Join(t.TempDir(), "test.log")
tempJSON := filepath.Join(t.TempDir(), "test.json")
dv := &codersdk.DeploymentValues{
Logging: codersdk.LoggingConfig{
Filter: []string{"foo", "baz"},
Human: serpent.String(tempFile),
JSON: serpent.String(tempJSON),
},
Verbose: true,
Trace: codersdk.TraceConfig{
Enable: true,
},
}
cmd := &serpent.Command{
Use: "test",
Handler: testHandler(t, clilog.FromDeploymentValues(dv)),
}
err := cmd.Invoke().Run()
require.NoError(t, err)
assertLogs(t, tempFile, infoLog, warnLog)
assertLogsJSON(t, tempJSON, info, infoLog, warn, warnLog)
})
})
t.Run("NotFound", func(t *testing.T) {
t.Parallel()
tempFile := filepath.Join(t.TempDir(), "doesnotexist", "test.log")
cmd := &serpent.Command{
Use: "test",
Handler: func(inv *serpent.Invocation) error {
logger, closeLog, err := clilog.New(
clilog.WithFilter("foo", "baz"),
clilog.WithHuman(tempFile),
clilog.WithVerbose(),
).Build(inv)
if err != nil {
return err
}
defer closeLog()
logger.Error(inv.Context(), "you will never see this")
return nil
},
}
err := cmd.Invoke().Run()
require.ErrorIs(t, err, fs.ErrNotExist)
})
}
var (
debug = "DEBUG"
info = "INFO"
warn = "WARN"
debugLog = "this is a debug message"
infoLog = "this is an info message"
warnLog = "this is a warning message"
filterLog = "this is an important debug message you want to see"
)
func testHandler(t testing.TB, opts ...clilog.Option) serpent.HandlerFunc {
t.Helper()
return func(inv *serpent.Invocation) error {
logger, closeLog, err := clilog.New(opts...).Build(inv)
if err != nil {
return err
}
defer closeLog()
logger.Debug(inv.Context(), debugLog)
logger.Info(inv.Context(), infoLog)
logger.Warn(inv.Context(), warnLog)
logger.Debug(inv.Context(), filterLog)
return nil
}
}
func assertLogs(t testing.TB, path string, expected ...string) {
t.Helper()
data, err := os.ReadFile(path)
require.NoError(t, err)
logs := strings.Split(strings.TrimSpace(string(data)), "\n")
if !assert.Len(t, logs, len(expected)) {
t.Logf(string(data))
t.FailNow()
}
for i, log := range logs {
require.Contains(t, log, expected[i])
}
}
func assertLogsJSON(t testing.TB, path string, levelExpected ...string) {
t.Helper()
data, err := os.ReadFile(path)
require.NoError(t, err)
if len(levelExpected)%2 != 0 {
t.Errorf("levelExpected must be a list of level-message pairs")
return
}
logs := strings.Split(strings.TrimSpace(string(data)), "\n")
if !assert.Len(t, logs, len(levelExpected)/2) {
t.Logf(string(data))
t.FailNow()
}
for i, log := range logs {
var entry struct {
Level string `json:"level"`
Message string `json:"msg"`
}
err := json.NewDecoder(strings.NewReader(log)).Decode(&entry)
require.NoError(t, err)
require.Equal(t, levelExpected[2*i], entry.Level)
require.Equal(t, levelExpected[2*i+1], entry.Message)
}
}
+2
View File
@@ -0,0 +1,2 @@
// Package clilog provides a fluent API for configuring structured logging.
package clilog
+18 -8
View File
@@ -20,16 +20,16 @@ import (
"cdr.dev/slog"
"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/v2/cli"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/config"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/provisioner/echo"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
// New creates a CLI instance with a configuration pointed to a
// temporary testing directory.
func New(t testing.TB, args ...string) (*clibase.Invocation, config.Root) {
func New(t testing.TB, args ...string) (*serpent.Invocation, config.Root) {
var root cli.RootCmd
cmd, err := root.Command(root.AGPL())
@@ -56,15 +56,15 @@ func (l *logWriter) Write(p []byte) (n int, err error) {
}
func NewWithCommand(
t testing.TB, cmd *clibase.Cmd, args ...string,
) (*clibase.Invocation, config.Root) {
t testing.TB, cmd *serpent.Command, args ...string,
) (*serpent.Invocation, config.Root) {
configDir := config.Root(t.TempDir())
// I really would like to fail test on error logs, but realistically, turning on by default
// in all our CLI tests is going to create a lot of flaky noise.
logger := slogtest.Make(t, &slogtest.Options{IgnoreErrors: true}).
Leveled(slog.LevelDebug).
Named("cli")
i := &clibase.Invocation{
i := &serpent.Invocation{
Command: cmd,
Args: append([]string{"--global-config", string(configDir)}, args...),
Stdin: io.LimitReader(nil, 0),
@@ -140,7 +140,11 @@ func extractTar(t *testing.T, data []byte, directory string) {
// Start runs the command in a goroutine and cleans it up when the test
// completed.
func Start(t *testing.T, inv *clibase.Invocation) {
func Start(t *testing.T, inv *serpent.Invocation) {
StartWithAssert(t, inv, nil)
}
func StartWithAssert(t *testing.T, inv *serpent.Invocation, assertCallback func(t *testing.T, err error)) { //nolint:revive
t.Helper()
closeCh := make(chan struct{})
@@ -155,6 +159,12 @@ func Start(t *testing.T, inv *clibase.Invocation) {
go func() {
defer close(closeCh)
err := waiter.Wait()
if assertCallback != nil {
assertCallback(t, err)
return
}
switch {
case errors.Is(err, context.Canceled):
return
@@ -165,7 +175,7 @@ func Start(t *testing.T, inv *clibase.Invocation) {
}
// Run runs the command and asserts that there is no error.
func Run(t *testing.T, inv *clibase.Invocation) {
func Run(t *testing.T, inv *serpent.Invocation) {
t.Helper()
err := inv.Run()
@@ -218,7 +228,7 @@ func (w *ErrorWaiter) RequireAs(want interface{}) {
// StartWithWaiter runs the command in a goroutine but returns the error instead
// of asserting it. This is useful for testing error cases.
func StartWithWaiter(t *testing.T, inv *clibase.Invocation) *ErrorWaiter {
func StartWithWaiter(t *testing.T, inv *serpent.Invocation) *ErrorWaiter {
t.Helper()
var (
+41 -32
View File
@@ -13,12 +13,12 @@ import (
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/config"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/coderd/database/dbtestutil"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
// UpdateGoldenFiles indicates golden files should be updated.
@@ -48,7 +48,7 @@ func DefaultCases() []CommandHelpCase {
// TestCommandHelp will test the help output of the given commands
// using golden files.
func TestCommandHelp(t *testing.T, getRoot func(t *testing.T) *clibase.Cmd, cases []CommandHelpCase) {
func TestCommandHelp(t *testing.T, getRoot func(t *testing.T) *serpent.Command, cases []CommandHelpCase) {
t.Parallel()
rootClient, replacements := prepareTestData(t)
@@ -87,40 +87,45 @@ ExtractCommandPathsLoop:
StartWithWaiter(t, inv.WithContext(ctx)).RequireSuccess()
actual := outBuf.Bytes()
if len(actual) == 0 {
t.Fatal("no output")
}
for k, v := range replacements {
actual = bytes.ReplaceAll(actual, []byte(k), []byte(v))
}
actual = NormalizeGoldenFile(t, actual)
goldenPath := filepath.Join("testdata", strings.Replace(tt.Name, " ", "_", -1)+".golden")
if *UpdateGoldenFiles {
t.Logf("update golden file for: %q: %s", tt.Name, goldenPath)
err := os.WriteFile(goldenPath, actual, 0o600)
require.NoError(t, err, "update golden file")
}
expected, err := os.ReadFile(goldenPath)
require.NoError(t, err, "read golden file, run \"make update-golden-files\" and commit the changes")
expected = NormalizeGoldenFile(t, expected)
require.Equal(
t, string(expected), string(actual),
"golden file mismatch: %s, run \"make update-golden-files\", verify and commit the changes",
goldenPath,
)
TestGoldenFile(t, tt.Name, outBuf.Bytes(), replacements)
})
}
}
// NormalizeGoldenFile replaces any strings that are system or timing dependent
// TestGoldenFile will test the given bytes slice input against the
// golden file with the given file name, optionally using the given replacements.
func TestGoldenFile(t *testing.T, fileName string, actual []byte, replacements map[string]string) {
if len(actual) == 0 {
t.Fatal("no output")
}
for k, v := range replacements {
actual = bytes.ReplaceAll(actual, []byte(k), []byte(v))
}
actual = normalizeGoldenFile(t, actual)
goldenPath := filepath.Join("testdata", strings.ReplaceAll(fileName, " ", "_")+".golden")
if *UpdateGoldenFiles {
t.Logf("update golden file for: %q: %s", fileName, goldenPath)
err := os.WriteFile(goldenPath, actual, 0o600)
require.NoError(t, err, "update golden file")
}
expected, err := os.ReadFile(goldenPath)
require.NoError(t, err, "read golden file, run \"make update-golden-files\" and commit the changes")
expected = normalizeGoldenFile(t, expected)
require.Equal(
t, string(expected), string(actual),
"golden file mismatch: %s, run \"make update-golden-files\", verify and commit the changes",
goldenPath,
)
}
// normalizeGoldenFile replaces any strings that are system or timing dependent
// with a placeholder so that the golden files can be compared with a simple
// equality check.
func NormalizeGoldenFile(t *testing.T, byt []byte) []byte {
func normalizeGoldenFile(t *testing.T, byt []byte) []byte {
// Replace any timestamps with a placeholder.
byt = timestampRegex.ReplaceAll(byt, []byte("[timestamp]"))
@@ -148,7 +153,7 @@ func NormalizeGoldenFile(t *testing.T, byt []byte) []byte {
return byt
}
func extractVisibleCommandPaths(cmdPath []string, cmds []*clibase.Cmd) [][]string {
func extractVisibleCommandPaths(cmdPath []string, cmds []*serpent.Command) [][]string {
var cmdPaths [][]string
for _, c := range cmds {
if c.Hidden {
@@ -167,7 +172,11 @@ func prepareTestData(t *testing.T) (*codersdk.Client, map[string]string) {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitLong)
defer cancel()
db, pubsub := dbtestutil.NewDB(t)
// This needs to be a fixed timezone because timezones increase the length
// of timestamp strings. The increased length can pad table formatting's
// and differ the table header spacings.
//nolint:gocritic
db, pubsub := dbtestutil.NewDB(t, dbtestutil.WithTimezone("UTC"))
rootClient := coderdtest.New(t, &coderdtest.Options{
Database: db,
Pubsub: pubsub,
+4 -4
View File
@@ -3,7 +3,7 @@ package clitest
import (
"testing"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/serpent"
)
// HandlersOK asserts that all commands have a handler.
@@ -11,11 +11,11 @@ import (
// non-root commands (like 'groups' or 'users'), a handler is required.
// These handlers are likely just the 'help' handler, but this must be
// explicitly set.
func HandlersOK(t *testing.T, cmd *clibase.Cmd) {
cmd.Walk(func(cmd *clibase.Cmd) {
func HandlersOK(t *testing.T, cmd *serpent.Command) {
cmd.Walk(func(cmd *serpent.Command) {
if cmd.Handler == nil {
// If you see this error, make the Handler a helper invoker.
// Handler: func(inv *clibase.Invocation) error {
// Handler: func(inv *serpent.Invocation) error {
// return inv.Command.HelpHandler(inv)
// },
t.Errorf("command %q has no handler, change to a helper invoker using: 'inv.Command.HelpHandler(inv)'", cmd.Name())
+83 -8
View File
@@ -2,13 +2,17 @@ package cliui
import (
"context"
"fmt"
"io"
"strconv"
"strings"
"time"
"github.com/google/uuid"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/tailnet"
)
var errAgentShuttingDown = xerrors.New("agent is shutting down")
@@ -200,28 +204,28 @@ func Agent(ctx context.Context, writer io.Writer, agentID uuid.UUID, opts AgentO
switch agent.LifecycleState {
case codersdk.WorkspaceAgentLifecycleReady:
sw.Complete(stage, agent.ReadyAt.Sub(*agent.StartedAt))
sw.Complete(stage, safeDuration(sw, agent.ReadyAt, agent.StartedAt))
case codersdk.WorkspaceAgentLifecycleStartTimeout:
sw.Fail(stage, 0)
sw.Log(time.Time{}, codersdk.LogLevelWarn, "Warning: A startup script timed out and your workspace may be incomplete.")
case codersdk.WorkspaceAgentLifecycleStartError:
sw.Fail(stage, agent.ReadyAt.Sub(*agent.StartedAt))
sw.Fail(stage, safeDuration(sw, agent.ReadyAt, agent.StartedAt))
// Use zero time (omitted) to separate these from the startup logs.
sw.Log(time.Time{}, codersdk.LogLevelWarn, "Warning: A startup script exited with an error and your workspace may be incomplete.")
sw.Log(time.Time{}, codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates#startup-script-exited-with-an-error"))
sw.Log(time.Time{}, codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates/troubleshooting#startup-script-exited-with-an-error"))
default:
switch {
case agent.LifecycleState.Starting():
// Use zero time (omitted) to separate these from the startup logs.
sw.Log(time.Time{}, codersdk.LogLevelWarn, "Notice: The startup scripts are still running and your workspace may be incomplete.")
sw.Log(time.Time{}, codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates#your-workspace-may-be-incomplete"))
sw.Log(time.Time{}, codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates/troubleshooting#your-workspace-may-be-incomplete"))
// Note: We don't complete or fail the stage here, it's
// intentionally left open to indicate this stage didn't
// complete.
case agent.LifecycleState.ShuttingDown():
// We no longer know if the startup script failed or not,
// but we need to tell the user something.
sw.Complete(stage, agent.ReadyAt.Sub(*agent.StartedAt))
sw.Complete(stage, safeDuration(sw, agent.ReadyAt, agent.StartedAt))
return errAgentShuttingDown
}
}
@@ -236,15 +240,15 @@ func Agent(ctx context.Context, writer io.Writer, agentID uuid.UUID, opts AgentO
stage := "The workspace agent lost connection"
sw.Start(stage)
sw.Log(time.Now(), codersdk.LogLevelWarn, "Wait for it to reconnect or restart your workspace.")
sw.Log(time.Now(), codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates#agent-connection-issues"))
sw.Log(time.Now(), codersdk.LogLevelWarn, troubleshootingMessage(agent, "https://coder.com/docs/v2/latest/templates/troubleshooting#agent-connection-issues"))
disconnectedAt := *agent.DisconnectedAt
disconnectedAt := agent.DisconnectedAt
for agent.Status == codersdk.WorkspaceAgentDisconnected {
if agent, err = fetch(); err != nil {
return xerrors.Errorf("fetch: %w", err)
}
}
sw.Complete(stage, agent.LastConnectedAt.Sub(disconnectedAt))
sw.Complete(stage, safeDuration(sw, agent.LastConnectedAt, disconnectedAt))
}
}
}
@@ -257,8 +261,79 @@ func troubleshootingMessage(agent codersdk.WorkspaceAgent, url string) string {
return m
}
// safeDuration returns a-b. If a or b is nil, it returns 0.
// This is because we often dereference a time pointer, which can
// cause a panic. These dereferences are used to calculate durations,
// which are not critical, and therefor should not break things
// when it fails.
// A panic has been observed in a test.
func safeDuration(sw *stageWriter, a, b *time.Time) time.Duration {
if a == nil || b == nil {
if sw != nil {
// Ideally the message includes which fields are <nil>, but you can
// use the surrounding log lines to figure that out. And passing more
// params makes this unwieldy.
sw.Log(time.Now(), codersdk.LogLevelWarn, "Warning: Failed to calculate duration from a time being <nil>.")
}
return 0
}
return a.Sub(*b)
}
type closeFunc func() error
func (c closeFunc) Close() error {
return c()
}
func PeerDiagnostics(w io.Writer, d tailnet.PeerDiagnostics) {
if d.PreferredDERP > 0 {
rn, ok := d.DERPRegionNames[d.PreferredDERP]
if !ok {
rn = "unknown"
}
_, _ = fmt.Fprintf(w, "✔ preferred DERP region: %d (%s)\n", d.PreferredDERP, rn)
} else {
_, _ = fmt.Fprint(w, "✘ not connected to DERP\n")
}
if d.SentNode {
_, _ = fmt.Fprint(w, "✔ sent local data to Coder networking coodinator\n")
} else {
_, _ = fmt.Fprint(w, "✘ have not sent local data to Coder networking coordinator\n")
}
if d.ReceivedNode != nil {
dp := d.ReceivedNode.DERP
dn := ""
// should be 127.3.3.40:N where N is the DERP region
ap := strings.Split(dp, ":")
if len(ap) == 2 {
dp = ap[1]
di, err := strconv.Atoi(dp)
if err == nil {
var ok bool
dn, ok = d.DERPRegionNames[di]
if ok {
dn = fmt.Sprintf("(%s)", dn)
} else {
dn = "(unknown)"
}
}
}
_, _ = fmt.Fprintf(w,
"✔ received remote agent data from Coder networking coordinator\n preferred DERP region: %s %s\n endpoints: %s\n",
dp, dn, strings.Join(d.ReceivedNode.Endpoints, ", "))
} else {
_, _ = fmt.Fprint(w, "✘ have not received remote agent data from Coder networking coordinator\n")
}
if !d.LastWireguardHandshake.IsZero() {
ago := time.Since(d.LastWireguardHandshake)
symbol := "✔"
// wireguard is supposed to refresh handshake on 5 minute intervals
if ago > 5*time.Minute {
symbol = "⚠"
}
_, _ = fmt.Fprintf(w, "%s Wireguard handshake %s ago\n", symbol, ago.Round(time.Second))
} else {
_, _ = fmt.Fprint(w, "✘ Wireguard is not connected\n")
}
}
+196 -5
View File
@@ -6,6 +6,7 @@ import (
"context"
"io"
"os"
"regexp"
"strings"
"sync/atomic"
"testing"
@@ -15,13 +16,15 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/xerrors"
"tailscale.com/tailcfg"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/clitest"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/coderd/util/ptr"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/tailnet"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
func TestAgent(t *testing.T) {
@@ -379,8 +382,8 @@ func TestAgent(t *testing.T) {
output := make(chan string, 100) // Buffered to avoid blocking, overflow is discarded.
logs := make(chan []codersdk.WorkspaceAgentLog, 1)
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
tc.opts.Fetch = func(_ context.Context, _ uuid.UUID) (codersdk.WorkspaceAgent, error) {
t.Log("iter", len(tc.iter))
var err error
@@ -447,8 +450,8 @@ func TestAgent(t *testing.T) {
t.Parallel()
var fetchCalled uint64
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
buf := bytes.Buffer{}
err := cliui.Agent(inv.Context(), &buf, uuid.Nil, cliui.AgentOptions{
FetchInterval: 10 * time.Millisecond,
@@ -476,3 +479,191 @@ func TestAgent(t *testing.T) {
require.NoError(t, cmd.Invoke().Run())
})
}
func TestPeerDiagnostics(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
diags tailnet.PeerDiagnostics
want []*regexp.Regexp // must be ordered, can omit lines
}{
{
name: "noPreferredDERP",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: make(map[int]string),
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Now(),
},
want: []*regexp.Regexp{
regexp.MustCompile("^✘ not connected to DERP$"),
},
},
{
name: "preferredDERP",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 23,
DERPRegionNames: map[int]string{
23: "testo",
},
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Now(),
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ preferred DERP region: 23 \(testo\)$`),
},
},
{
name: "sentNode",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ sent local data to Coder networking coodinator$`),
},
},
{
name: "didntSendNode",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✘ have not sent local data to Coder networking coordinator$`),
},
},
{
name: "receivedNodeDERPOKNoEndpoints",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{999: "Embedded"},
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ received remote agent data from Coder networking coordinator$`),
regexp.MustCompile(`preferred DERP region: 999 \(Embedded\)$`),
regexp.MustCompile(`endpoints: $`),
},
},
{
name: "receivedNodeDERPUnknownNoEndpoints",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: true,
ReceivedNode: &tailcfg.Node{DERP: "127.3.3.40:999"},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ received remote agent data from Coder networking coordinator$`),
regexp.MustCompile(`preferred DERP region: 999 \(unknown\)$`),
regexp.MustCompile(`endpoints: $`),
},
},
{
name: "receivedNodeEndpointsNoDERP",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{999: "Embedded"},
SentNode: true,
ReceivedNode: &tailcfg.Node{Endpoints: []string{"99.88.77.66:4555", "33.22.11.0:3444"}},
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ received remote agent data from Coder networking coordinator$`),
regexp.MustCompile(`preferred DERP region:\s*$`),
regexp.MustCompile(`endpoints: 99\.88\.77\.66:4555, 33\.22\.11\.0:3444$`),
},
},
{
name: "didntReceiveNode",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: nil,
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✘ have not received remote agent data from Coder networking coordinator$`),
},
},
{
name: "noWireguardHandshake",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: nil,
LastWireguardHandshake: time.Time{},
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✘ Wireguard is not connected$`),
},
},
{
name: "wireguardHandshakeRecent",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: nil,
LastWireguardHandshake: time.Now().Add(-5 * time.Second),
},
want: []*regexp.Regexp{
regexp.MustCompile(`^✔ Wireguard handshake \d+s ago$`),
},
},
{
name: "wireguardHandshakeOld",
diags: tailnet.PeerDiagnostics{
PreferredDERP: 0,
DERPRegionNames: map[int]string{},
SentNode: false,
ReceivedNode: nil,
LastWireguardHandshake: time.Now().Add(-450 * time.Second), // 7m30s
},
want: []*regexp.Regexp{
regexp.MustCompile(`^⚠ Wireguard handshake 7m\d+s ago$`),
},
},
}
for _, tc := range testCases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
r, w := io.Pipe()
go func() {
defer w.Close()
cliui.PeerDiagnostics(w, tc.diags)
}()
s := bufio.NewScanner(r)
i := 0
got := make([]string, 0)
for s.Scan() {
got = append(got, s.Text())
if i < len(tc.want) {
reg := tc.want[i]
if reg.Match(s.Bytes()) {
i++
}
}
}
if i < len(tc.want) {
t.Logf("failed to match regexp: %s\ngot:\n%s", tc.want[i].String(), strings.Join(got, "\n"))
t.FailNow()
}
})
}
}
+21
View File
@@ -0,0 +1,21 @@
package cliui
import (
"fmt"
"github.com/coder/pretty"
"github.com/coder/serpent"
)
func DeprecationWarning(message string) serpent.MiddlewareFunc {
return func(next serpent.HandlerFunc) serpent.HandlerFunc {
return func(i *serpent.Invocation) error {
_, _ = fmt.Fprintln(i.Stdout, "\n"+pretty.Sprint(DefaultStyles.Wrap,
pretty.Sprint(
DefaultStyles.Warn,
"DEPRECATION WARNING: This command will be removed in a future release."+"\n"+message+"\n"),
))
return next(i)
}
}
}
+3 -3
View File
@@ -8,11 +8,11 @@ import (
"github.com/stretchr/testify/assert"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
func TestExternalAuth(t *testing.T) {
@@ -22,8 +22,8 @@ func TestExternalAuth(t *testing.T) {
defer cancel()
ptty := ptytest.New(t)
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
var fetched atomic.Bool
return cliui.ExternalAuth(inv.Context(), inv.Stdout, cliui.ExternalAuthOptions{
Fetch: func(ctx context.Context) ([]codersdk.TemplateVersionExternalAuth, error) {
+8 -8
View File
@@ -1,8 +1,8 @@
package cliui
import (
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
var defaultQuery = "owner:me"
@@ -11,12 +11,12 @@ var defaultQuery = "owner:me"
// and allows easy integration to a CLI command.
// Example usage:
//
// func (r *RootCmd) MyCmd() *clibase.Cmd {
// func (r *RootCmd) MyCmd() *serpent.Command {
// var (
// filter cliui.WorkspaceFilter
// ...
// )
// cmd := &clibase.Cmd{
// cmd := &serpent.Command{
// ...
// }
// filter.AttachOptions(&cmd.Options)
@@ -44,20 +44,20 @@ func (w *WorkspaceFilter) Filter() codersdk.WorkspaceFilter {
return f
}
func (w *WorkspaceFilter) AttachOptions(opts *clibase.OptionSet) {
func (w *WorkspaceFilter) AttachOptions(opts *serpent.OptionSet) {
*opts = append(*opts,
clibase.Option{
serpent.Option{
Flag: "all",
FlagShorthand: "a",
Description: "Specifies whether all workspaces will be listed or not.",
Value: clibase.BoolOf(&w.all),
Value: serpent.BoolOf(&w.all),
},
clibase.Option{
serpent.Option{
Flag: "search",
Description: "Search for a workspace with a query.",
Default: defaultQuery,
Value: clibase.StringOf(&w.searchQuery),
Value: serpent.StringOf(&w.searchQuery),
},
)
}
+12 -12
View File
@@ -9,12 +9,12 @@ import (
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/serpent"
)
type OutputFormat interface {
ID() string
AttachOptions(opts *clibase.OptionSet)
AttachOptions(opts *serpent.OptionSet)
Format(ctx context.Context, data any) (string, error)
}
@@ -49,7 +49,7 @@ func NewOutputFormatter(formats ...OutputFormat) *OutputFormatter {
// AttachOptions attaches the --output flag to the given command, and any
// additional flags required by the output formatters.
func (f *OutputFormatter) AttachOptions(opts *clibase.OptionSet) {
func (f *OutputFormatter) AttachOptions(opts *serpent.OptionSet) {
for _, format := range f.formats {
format.AttachOptions(opts)
}
@@ -60,11 +60,11 @@ func (f *OutputFormatter) AttachOptions(opts *clibase.OptionSet) {
}
*opts = append(*opts,
clibase.Option{
serpent.Option{
Flag: "output",
FlagShorthand: "o",
Default: f.formats[0].ID(),
Value: clibase.StringOf(&f.formatID),
Value: serpent.StringOf(&f.formatID),
Description: "Output format. Available formats: " + strings.Join(formatNames, ", ") + ".",
},
)
@@ -106,7 +106,7 @@ func TableFormat(out any, defaultColumns []string) OutputFormat {
}
// Get the list of table column headers.
headers, defaultSort, err := typeToTableHeaders(v.Type().Elem())
headers, defaultSort, err := typeToTableHeaders(v.Type().Elem(), true)
if err != nil {
panic("parse table headers: " + err.Error())
}
@@ -129,13 +129,13 @@ func (*tableFormat) ID() string {
}
// AttachOptions implements OutputFormat.
func (f *tableFormat) AttachOptions(opts *clibase.OptionSet) {
func (f *tableFormat) AttachOptions(opts *serpent.OptionSet) {
*opts = append(*opts,
clibase.Option{
serpent.Option{
Flag: "column",
FlagShorthand: "c",
Default: strings.Join(f.defaultColumns, ","),
Value: clibase.StringArrayOf(&f.columns),
Value: serpent.StringArrayOf(&f.columns),
Description: "Columns to display in table output. Available columns: " + strings.Join(f.allColumns, ", ") + ".",
},
)
@@ -161,7 +161,7 @@ func (jsonFormat) ID() string {
}
// AttachOptions implements OutputFormat.
func (jsonFormat) AttachOptions(_ *clibase.OptionSet) {}
func (jsonFormat) AttachOptions(_ *serpent.OptionSet) {}
// Format implements OutputFormat.
func (jsonFormat) Format(_ context.Context, data any) (string, error) {
@@ -187,7 +187,7 @@ func (textFormat) ID() string {
return "text"
}
func (textFormat) AttachOptions(_ *clibase.OptionSet) {}
func (textFormat) AttachOptions(_ *serpent.OptionSet) {}
func (textFormat) Format(_ context.Context, data any) (string, error) {
return fmt.Sprintf("%s", data), nil
@@ -213,7 +213,7 @@ func (d *DataChangeFormat) ID() string {
return d.format.ID()
}
func (d *DataChangeFormat) AttachOptions(opts *clibase.OptionSet) {
func (d *DataChangeFormat) AttachOptions(opts *serpent.OptionSet) {
d.format.AttachOptions(opts)
}
+7 -7
View File
@@ -8,13 +8,13 @@ import (
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/serpent"
)
type format struct {
id string
attachOptionsFn func(opts *clibase.OptionSet)
attachOptionsFn func(opts *serpent.OptionSet)
formatFn func(ctx context.Context, data any) (string, error)
}
@@ -24,7 +24,7 @@ func (f *format) ID() string {
return f.id
}
func (f *format) AttachOptions(opts *clibase.OptionSet) {
func (f *format) AttachOptions(opts *serpent.OptionSet) {
if f.attachOptionsFn != nil {
f.attachOptionsFn(opts)
}
@@ -85,12 +85,12 @@ func Test_OutputFormatter(t *testing.T) {
cliui.JSONFormat(),
&format{
id: "foo",
attachOptionsFn: func(opts *clibase.OptionSet) {
opts.Add(clibase.Option{
attachOptionsFn: func(opts *serpent.OptionSet) {
opts.Add(serpent.Option{
Name: "foo",
Flag: "foo",
FlagShorthand: "f",
Value: clibase.DiscardValue,
Value: serpent.DiscardValue,
Description: "foo flag 1234",
})
},
@@ -101,7 +101,7 @@ func Test_OutputFormatter(t *testing.T) {
},
)
cmd := &clibase.Cmd{}
cmd := &serpent.Command{}
f.AttachOptions(&cmd.Options)
fs := cmd.Options.FlagSet()
+10 -5
View File
@@ -5,12 +5,12 @@ import (
"fmt"
"strings"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/pretty"
"github.com/coder/serpent"
)
func RichParameter(inv *clibase.Invocation, templateVersionParameter codersdk.TemplateVersionParameter) (string, error) {
func RichParameter(inv *serpent.Invocation, templateVersionParameter codersdk.TemplateVersionParameter, defaultOverrides map[string]string) (string, error) {
label := templateVersionParameter.Name
if templateVersionParameter.DisplayName != "" {
label = templateVersionParameter.DisplayName
@@ -26,6 +26,11 @@ func RichParameter(inv *clibase.Invocation, templateVersionParameter codersdk.Te
_, _ = fmt.Fprintln(inv.Stdout, " "+strings.TrimSpace(strings.Join(strings.Split(templateVersionParameter.DescriptionPlaintext, "\n"), "\n "))+"\n")
}
defaultValue := templateVersionParameter.DefaultValue
if v, ok := defaultOverrides[templateVersionParameter.Name]; ok {
defaultValue = v
}
var err error
var value string
if templateVersionParameter.Type == "list(string)" {
@@ -58,7 +63,7 @@ func RichParameter(inv *clibase.Invocation, templateVersionParameter codersdk.Te
var richParameterOption *codersdk.TemplateVersionParameterOption
richParameterOption, err = RichSelect(inv, RichSelectOptions{
Options: templateVersionParameter.Options,
Default: templateVersionParameter.DefaultValue,
Default: defaultValue,
HideSearch: true,
})
if err == nil {
@@ -69,7 +74,7 @@ func RichParameter(inv *clibase.Invocation, templateVersionParameter codersdk.Te
} else {
text := "Enter a value"
if !templateVersionParameter.Required {
text += fmt.Sprintf(" (default: %q)", templateVersionParameter.DefaultValue)
text += fmt.Sprintf(" (default: %q)", defaultValue)
}
text += ":"
@@ -87,7 +92,7 @@ func RichParameter(inv *clibase.Invocation, templateVersionParameter codersdk.Te
// If they didn't specify anything, use the default value if set.
if len(templateVersionParameter.Options) == 0 && value == "" {
value = templateVersionParameter.DefaultValue
value = defaultValue
}
return value, nil
+7 -7
View File
@@ -13,8 +13,8 @@ import (
"github.com/mattn/go-isatty"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/pretty"
"github.com/coder/serpent"
)
// PromptOptions supply a set of options to the prompt.
@@ -30,13 +30,13 @@ const skipPromptFlag = "yes"
// SkipPromptOption adds a "--yes/-y" flag to the cmd that can be used to skip
// prompts.
func SkipPromptOption() clibase.Option {
return clibase.Option{
func SkipPromptOption() serpent.Option {
return serpent.Option{
Flag: skipPromptFlag,
FlagShorthand: "y",
Description: "Bypass prompts.",
// Discard
Value: clibase.BoolOf(new(bool)),
Value: serpent.BoolOf(new(bool)),
}
}
@@ -46,7 +46,7 @@ const (
)
// Prompt asks the user for input.
func Prompt(inv *clibase.Invocation, opts PromptOptions) (string, error) {
func Prompt(inv *serpent.Invocation, opts PromptOptions) (string, error) {
// If the cmd has a "yes" flag for skipping confirm prompts, honor it.
// If it's not a "Confirm" prompt, then don't skip. As the default value of
// "yes" makes no sense.
@@ -71,9 +71,9 @@ func Prompt(inv *clibase.Invocation, opts PromptOptions) (string, error) {
} else {
renderedNo = Bold(ConfirmNo)
}
pretty.Fprintf(inv.Stdout, DefaultStyles.Placeholder, "(%s/%s) ", renderedYes, renderedNo)
_, _ = fmt.Fprintf(inv.Stdout, "(%s/%s) ", renderedYes, renderedNo)
} else if opts.Default != "" {
_, _ = fmt.Fprint(inv.Stdout, pretty.Sprint(DefaultStyles.Placeholder, "("+opts.Default+") "))
_, _ = fmt.Fprintf(inv.Stdout, "(%s) ", pretty.Sprint(DefaultStyles.Placeholder, opts.Default))
}
interrupt := make(chan os.Signal, 1)
+7 -7
View File
@@ -11,11 +11,11 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/pty"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/coder/v2/testutil"
"github.com/coder/serpent"
)
func TestPrompt(t *testing.T) {
@@ -77,7 +77,7 @@ func TestPrompt(t *testing.T) {
resp, err := newPrompt(ptty, cliui.PromptOptions{
Text: "ShouldNotSeeThis",
IsConfirm: true,
}, func(inv *clibase.Invocation) {
}, func(inv *serpent.Invocation) {
inv.Command.Options = append(inv.Command.Options, cliui.SkipPromptOption())
inv.Args = []string{"-y"}
})
@@ -145,10 +145,10 @@ func TestPrompt(t *testing.T) {
})
}
func newPrompt(ptty *ptytest.PTY, opts cliui.PromptOptions, invOpt func(inv *clibase.Invocation)) (string, error) {
func newPrompt(ptty *ptytest.PTY, opts cliui.PromptOptions, invOpt func(inv *serpent.Invocation)) (string, error) {
value := ""
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
var err error
value, err = cliui.Prompt(inv, opts)
return err
@@ -210,8 +210,8 @@ func TestPasswordTerminalState(t *testing.T) {
// nolint:unused
func passwordHelper() {
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
cliui.Prompt(inv, cliui.PromptOptions{
Text: "Password:",
Secret: true,
+35 -5
View File
@@ -54,6 +54,11 @@ func (err *ProvisionerJobError) Error() string {
return err.Message
}
const (
ProvisioningStateQueued = "Queued"
ProvisioningStateRunning = "Running"
)
// ProvisionerJob renders a provisioner job with interactive cancellation.
func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOptions) error {
if opts.FetchInterval == 0 {
@@ -63,8 +68,9 @@ func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOption
defer cancelFunc()
var (
currentStage = "Queued"
currentStage = ProvisioningStateQueued
currentStageStartedAt = time.Now().UTC()
currentQueuePos = -1
errChan = make(chan error, 1)
job codersdk.ProvisionerJob
@@ -74,7 +80,20 @@ func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOption
sw := &stageWriter{w: wr, verbose: opts.Verbose, silentLogs: opts.Silent}
printStage := func() {
sw.Start(currentStage)
out := currentStage
if currentStage == ProvisioningStateQueued && currentQueuePos > 0 {
var queuePos string
if currentQueuePos == 1 {
queuePos = "next"
} else {
queuePos = fmt.Sprintf("position: %d", currentQueuePos)
}
out = pretty.Sprintf(DefaultStyles.Warn, "%s (%s)", currentStage, queuePos)
}
sw.Start(out)
}
updateStage := func(stage string, startedAt time.Time) {
@@ -103,15 +122,26 @@ func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOption
errChan <- xerrors.Errorf("fetch: %w", err)
return
}
if job.QueuePosition != currentQueuePos {
initialState := currentQueuePos == -1
currentQueuePos = job.QueuePosition
// Print an update when the queue position changes, but:
// - not initially, because the stage is printed at startup
// - not when we're first in the queue, because it's redundant
if !initialState && currentQueuePos != 0 {
printStage()
}
}
if job.StartedAt == nil {
return
}
if currentStage != "Queued" {
if currentStage != ProvisioningStateQueued {
// If another stage is already running, there's no need
// for us to notify the user we're running!
return
}
updateStage("Running", *job.StartedAt)
updateStage(ProvisioningStateRunning, *job.StartedAt)
}
if opts.Cancel != nil {
@@ -143,8 +173,8 @@ func ProvisionerJob(ctx context.Context, wr io.Writer, opts ProvisionerJobOption
}
// The initial stage needs to print after the signal handler has been registered.
printStage()
updateJob()
printStage()
logs, closer, err := opts.Logs()
if err != nil {
+120 -26
View File
@@ -2,8 +2,10 @@ package cliui_test
import (
"context"
"fmt"
"io"
"os"
"regexp"
"runtime"
"sync"
"testing"
@@ -11,11 +13,13 @@ import (
"github.com/stretchr/testify/assert"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/testutil"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/coderd/database/dbtime"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/serpent"
)
// This cannot be ran in parallel because it uses a signal.
@@ -25,7 +29,11 @@ func TestProvisionerJob(t *testing.T) {
t.Parallel()
test := newProvisionerJob(t)
go func() {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancel()
testutil.Go(t, func() {
<-test.Next
test.JobMutex.Lock()
test.Job.Status = codersdk.ProvisionerJobRunning
@@ -39,20 +47,26 @@ func TestProvisionerJob(t *testing.T) {
test.Job.CompletedAt = &now
close(test.Logs)
test.JobMutex.Unlock()
}()
test.PTY.ExpectMatch("Queued")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Queued")
test.PTY.ExpectMatch("Running")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Running")
})
testutil.Eventually(ctx, t, func(ctx context.Context) (done bool) {
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.PTY.ExpectMatch(cliui.ProvisioningStateRunning)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateRunning)
return true
}, testutil.IntervalFast)
})
t.Run("Stages", func(t *testing.T) {
t.Parallel()
test := newProvisionerJob(t)
go func() {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancel()
testutil.Go(t, func() {
<-test.Next
test.JobMutex.Lock()
test.Job.Status = codersdk.ProvisionerJobRunning
@@ -70,13 +84,86 @@ func TestProvisionerJob(t *testing.T) {
test.Job.CompletedAt = &now
close(test.Logs)
test.JobMutex.Unlock()
}()
test.PTY.ExpectMatch("Queued")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Queued")
test.PTY.ExpectMatch("Something")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Something")
})
testutil.Eventually(ctx, t, func(ctx context.Context) (done bool) {
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.PTY.ExpectMatch("Something")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Something")
return true
}, testutil.IntervalFast)
})
t.Run("Queue Position", func(t *testing.T) {
t.Parallel()
stage := cliui.ProvisioningStateQueued
tests := []struct {
name string
queuePos int
expected string
}{
{
name: "first",
queuePos: 0,
expected: fmt.Sprintf("%s$", stage),
},
{
name: "next",
queuePos: 1,
expected: fmt.Sprintf(`%s %s$`, stage, regexp.QuoteMeta("(next)")),
},
{
name: "other",
queuePos: 4,
expected: fmt.Sprintf(`%s %s$`, stage, regexp.QuoteMeta("(position: 4)")),
},
}
for _, tc := range tests {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
test := newProvisionerJob(t)
test.JobMutex.Lock()
test.Job.QueuePosition = tc.queuePos
test.Job.QueueSize = tc.queuePos
test.JobMutex.Unlock()
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancel()
testutil.Go(t, func() {
<-test.Next
test.JobMutex.Lock()
test.Job.Status = codersdk.ProvisionerJobRunning
now := dbtime.Now()
test.Job.StartedAt = &now
test.JobMutex.Unlock()
<-test.Next
test.JobMutex.Lock()
test.Job.Status = codersdk.ProvisionerJobSucceeded
now = dbtime.Now()
test.Job.CompletedAt = &now
close(test.Logs)
test.JobMutex.Unlock()
})
testutil.Eventually(ctx, t, func(ctx context.Context) (done bool) {
test.PTY.ExpectRegexMatch(tc.expected)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued) // step completed
test.PTY.ExpectMatch(cliui.ProvisioningStateRunning)
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateRunning)
return true
}, testutil.IntervalFast)
})
}
})
// This cannot be ran in parallel because it uses a signal.
@@ -90,7 +177,11 @@ func TestProvisionerJob(t *testing.T) {
}
test := newProvisionerJob(t)
go func() {
ctx, cancel := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancel()
testutil.Go(t, func() {
<-test.Next
currentProcess, err := os.FindProcess(os.Getpid())
assert.NoError(t, err)
@@ -103,12 +194,15 @@ func TestProvisionerJob(t *testing.T) {
test.Job.CompletedAt = &now
close(test.Logs)
test.JobMutex.Unlock()
}()
test.PTY.ExpectMatch("Queued")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Gracefully canceling")
test.Next <- struct{}{}
test.PTY.ExpectMatch("Queued")
})
testutil.Eventually(ctx, t, func(ctx context.Context) (done bool) {
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
test.Next <- struct{}{}
test.PTY.ExpectMatch("Gracefully canceling")
test.Next <- struct{}{}
test.PTY.ExpectMatch(cliui.ProvisioningStateQueued)
return true
}, testutil.IntervalFast)
})
}
@@ -127,8 +221,8 @@ func newProvisionerJob(t *testing.T) provisionerJobTest {
}
jobLock := sync.Mutex{}
logs := make(chan codersdk.ProvisionerJobLog, 1)
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
return cliui.ProvisionerJob(inv.Context(), inv.Stdout, cliui.ProvisionerJobOptions{
FetchInterval: time.Millisecond,
Fetch: func() (codersdk.ProvisionerJob, error) {
+4 -4
View File
@@ -10,8 +10,8 @@ import (
"github.com/AlecAivazis/survey/v2/terminal"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/serpent"
)
func init() {
@@ -68,7 +68,7 @@ type RichSelectOptions struct {
}
// RichSelect displays a list of user options including name and description.
func RichSelect(inv *clibase.Invocation, richOptions RichSelectOptions) (*codersdk.TemplateVersionParameterOption, error) {
func RichSelect(inv *serpent.Invocation, richOptions RichSelectOptions) (*codersdk.TemplateVersionParameterOption, error) {
opts := make([]string, len(richOptions.Options))
var defaultOpt string
for i, option := range richOptions.Options {
@@ -102,7 +102,7 @@ func RichSelect(inv *clibase.Invocation, richOptions RichSelectOptions) (*coders
}
// Select displays a list of user options.
func Select(inv *clibase.Invocation, opts SelectOptions) (string, error) {
func Select(inv *serpent.Invocation, opts SelectOptions) (string, error) {
// The survey library used *always* fails when testing on Windows,
// as it requires a live TTY (can't be a conpty). We should fork
// this library to add a dummy fallback, that simply reads/writes
@@ -138,7 +138,7 @@ func Select(inv *clibase.Invocation, opts SelectOptions) (string, error) {
return value, err
}
func MultiSelect(inv *clibase.Invocation, items []string) ([]string, error) {
func MultiSelect(inv *serpent.Invocation, items []string) ([]string, error) {
// Similar hack is applied to Select()
if flag.Lookup("test.v") != nil {
return items, nil
+7 -7
View File
@@ -6,10 +6,10 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/cli/clibase"
"github.com/coder/coder/v2/cli/cliui"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/pty/ptytest"
"github.com/coder/serpent"
)
func TestSelect(t *testing.T) {
@@ -31,8 +31,8 @@ func TestSelect(t *testing.T) {
func newSelect(ptty *ptytest.PTY, opts cliui.SelectOptions) (string, error) {
value := ""
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
var err error
value, err = cliui.Select(inv, opts)
return err
@@ -72,8 +72,8 @@ func TestRichSelect(t *testing.T) {
func newRichSelect(ptty *ptytest.PTY, opts cliui.RichSelectOptions) (string, error) {
value := ""
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
richOption, err := cliui.RichSelect(inv, opts)
if err == nil {
value = richOption.Value
@@ -105,8 +105,8 @@ func TestMultiSelect(t *testing.T) {
func newMultiSelect(ptty *ptytest.PTY, items []string) ([]string, error) {
var values []string
cmd := &clibase.Cmd{
Handler: func(inv *clibase.Invocation) error {
cmd := &serpent.Command{
Handler: func(inv *serpent.Invocation) error {
selectedItems, err := cliui.MultiSelect(inv, items)
if err == nil {
values = selectedItems
+17 -4
View File
@@ -70,7 +70,7 @@ func DisplayTable(out any, sort string, filterColumns []string) (string, error)
}
// Get the list of table column headers.
headersRaw, defaultSort, err := typeToTableHeaders(v.Type().Elem())
headersRaw, defaultSort, err := typeToTableHeaders(v.Type().Elem(), true)
if err != nil {
return "", xerrors.Errorf("get table headers recursively for type %q: %w", v.Type().Elem().String(), err)
}
@@ -230,7 +230,11 @@ func isStructOrStructPointer(t reflect.Type) bool {
// typeToTableHeaders converts a type to a slice of column names. If the given
// type is invalid (not a struct or a pointer to a struct, has invalid table
// tags, etc.), an error is returned.
func typeToTableHeaders(t reflect.Type) ([]string, string, error) {
//
// requireDefault is only needed for the root call. This is recursive, so nested
// structs do not need the default sort name.
// nolint:revive
func typeToTableHeaders(t reflect.Type, requireDefault bool) ([]string, string, error) {
if !isStructOrStructPointer(t) {
return nil, "", xerrors.Errorf("typeToTableHeaders called with a non-struct or a non-pointer-to-a-struct type")
}
@@ -246,6 +250,12 @@ func typeToTableHeaders(t reflect.Type) ([]string, string, error) {
if err != nil {
return nil, "", xerrors.Errorf("parse struct tags for field %q in type %q: %w", field.Name, t.String(), err)
}
if name == "" && (recursive && skip) {
return nil, "", xerrors.Errorf("a name is required for the field %q. "+
"recursive_line will ensure this is never shown to the user, but is still needed", field.Name)
}
// If recurse and skip is set, the name is intentionally empty.
if name == "" {
continue
}
@@ -262,7 +272,7 @@ func typeToTableHeaders(t reflect.Type) ([]string, string, error) {
return nil, "", xerrors.Errorf("field %q in type %q is marked as recursive but does not contain a struct or a pointer to a struct", field.Name, t.String())
}
childNames, _, err := typeToTableHeaders(fieldType)
childNames, defaultSort, err := typeToTableHeaders(fieldType, false)
if err != nil {
return nil, "", xerrors.Errorf("get child field header names for field %q in type %q: %w", field.Name, fieldType.String(), err)
}
@@ -273,13 +283,16 @@ func typeToTableHeaders(t reflect.Type) ([]string, string, error) {
}
headers = append(headers, fullName)
}
if defaultSortName == "" {
defaultSortName = defaultSort
}
continue
}
headers = append(headers, name)
}
if defaultSortName == "" {
if defaultSortName == "" && requireDefault {
return nil, "", xerrors.Errorf("no field marked as default_sort in type %q", t.String())
}
+2 -2
View File
@@ -46,12 +46,12 @@ type tableTest2 struct {
type tableTest3 struct {
NotIncluded string // no table tag
Sub tableTest2 `table:"inner,recursive,default_sort"`
Sub tableTest2 `table:"inner,recursive"`
}
type tableTest4 struct {
Inline tableTest2 `table:"ignored,recursive_inline"`
SortField string `table:"sort_field,default_sort"`
SortField string `table:"sort_field"`
}
func Test_DisplayTable(t *testing.T) {
+12 -2
View File
@@ -4,6 +4,7 @@ import (
"io"
"os"
"path/filepath"
"strings"
"github.com/kirsle/configdir"
"golang.org/x/xerrors"
@@ -69,6 +70,14 @@ func (r Root) PostgresPort() File {
// File provides convenience methods for interacting with *os.File.
type File string
func (f File) Exists() bool {
if f == "" {
return false
}
_, err := os.Stat(string(f))
return err == nil
}
// Delete deletes the file.
func (f File) Delete() error {
if f == "" {
@@ -85,13 +94,14 @@ func (f File) Write(s string) error {
return write(string(f), 0o600, []byte(s))
}
// Read reads the file to a string.
// Read reads the file to a string. All leading and trailing whitespace
// is removed.
func (f File) Read() (string, error) {
if f == "" {
return "", xerrors.Errorf("empty file path")
}
byt, err := read(string(f))
return string(byt), err
return strings.TrimSpace(string(byt)), err
}
// open opens a file in the configuration directory,

Some files were not shown because too many files have changed in this diff Show More