cloudflare/pint

Public

mirrored from https://github.com/cloudflare/pintAvailable

CodeCommitsIssuesPull requestsActionsInsightsSecurity
v0.3.0

Branches

Tags

  • No tags available.
0Branches0Tags
Go to file
Add file
Code

Clone

HTTPS

Download ZIP

docs/CONFIGURATION.md

801lines · modecode

1# Configuration syntax
2
3**NOTE** all regex pattern are anchored.
4
5## CI
6
7Configure continuous integration environments.
8
9Syntax:
10
11```JS
12ci {
13 include = [ "(.*)", ... ]
14 maxCommits = 20
15 baseBranch = "master"
16}
17```
18
19- `include` - list of file patters to checks when running checks. Only files
20 matching those regex rules will be checked, other modified files will be ignored.
21- `maxCommits` - by default pint will try to find all commits on current branch,
22 this requires full git history to be present, if we have a shallow clone this
23 might fail to find only current branch commits and give us a huge list.
24 If the number of commits returned by branch discovery is more than `maxCommits`
25 then pint will fail to run.
26- `baseBranch` - base branch to compare `HEAD` commit with when calculating the list
27 of commits to check.
28
29## Repository
30
31Configure supported code hosting repository, used for reporting PR checks from CI
32back to the repository, to be displayed in the PR UI.
33Currently only supports [BitBucket](https://bitbucket.org/) and [GitHub](https://github.com/).
34
35**NOTE**: BitBucket integration requires `BITBUCKET_AUTH_TOKEN` environment variable
36to be set. It should contain a personal access token used to authenticate with the API.
37
38**NOTE**: GitHub integration requires `GITHUB_AUTH_TOKEN` environment variable
39to be set to a personal access key that can access your repository. Also, `GITHUB_PULL_REQUEST_NUMBER`
40environment variable needs to point to the pull request number which will be used whilst
41submitting comments.
42
43Syntax:
44
45```JS
46repository {
47 bitbucket {
48 uri = "https://..."
49 timeout = "30s"
50 project = "..."
51 repository = "..."
52 }
53}
54```
55
56- `bitbucket:uri` - base URI of this repository, will be used for HTTP
57 requests to the BitBucket API.
58- `bitbucket:timeout` - timeout to be used for API requests.
59- `bitbucket:project` - name of the BitBucket project for this repository.
60- `bitbucket:repository` - name of the BitBucket repository.
61
62```JS
63repository {
64 github {
65 uri = "https://..."
66 timeout = "30s"
67 owner = "..."
68 repo = "..."
69 }
70}
71```
72
73- `github:baseuri` - base URI of GitHub or GitHub enterprise, will be used for HTTP requests to the GitHub API.
74- `github:uploaduri` - upload URI of GitHub or GitHub enterprise, will be used for HTTP requests to the GitHub API.
75
76If `github:baseuri` _or_ `github:uploaduri` are not specified then [GitHub](https://github.com) will be used.
77
78- `github:timeout` - timeout to be used for API requests;
79- `github:owner` - name of the GitHub owner i.e. the first part that comes before the repository's name in the URI;
80- `github:repo` - name of the GitHub repository (e.g. `monitoring`).
81
82## Prometheus servers
83
84Some checks work by querying a running Prometheus instance to verify if
85metrics used in rules are present. If you want to use those checks then you
86first need to define one or more Prometheus servers.
87
88Syntax:
89
90```JS
91prometheus "$name" {
92 uri = "https://..."
93 timeout = "60s"
94 paths = ["...", ...]
95}
96```
97
98- `$name` - each defined server should have a unique name that can be used in check
99 definitions.
100- `uri` - base URI of this Prometheus server, used for API requests and queries.
101- `timeout` - timeout to be used for API requests.
102- `paths` - optional path filter, if specified only paths matching one of listed regex
103 patterns will use this Prometheus server for checks.
104
105Example:
106
107```JS
108prometheus "prod" {
109 uri = "https://prometheus-prod.example.com"
110 timeout = "60s"
111}
112
113prometheus "dev" {
114 uri = "https://prometheus-dev.example.com"
115 timeout = "30s"
116 paths = [ "alerts/test/.*" ]
117}
118```
119
120## Matching rules to checks
121
122Most checks, except basic syntax verification, requires some configuration to decide
123which checks to run against which files and rules.
124
125Syntax:
126
127```JS
128rule {
129 match {
130 path = "(.+)"
131 name = "(.+)"
132 kind = "alerting|recording"
133 annotation "(.*)" {
134 value = "(.*)"
135 }
136 label "(.*)" {
137 value = "(.*)"
138 }
139 }
140 ignore {
141 path = "(.+)"
142 name = "(.+)"
143 kind = "alerting|recording"
144 annotation "(.*)" {
145 value = "(.*)"
146 }
147 label "(.*)" {
148 value = "(.*)"
149 }
150 }
151
152 [ check definition ]
153 ...
154 [ check definition ]
155}
156```
157
158- `match:path` - only files matching this pattern will be checked by this rule
159- `match:name` - only rules with names (`record` for recording rules and `alert` for alerting
160 rules) matching this pattern will be checked rule
161- `match:kind` - optional rule type filter, only rule of this type will be checked
162- `match:annotation` - optional annotation filter, only alert rules with at least one
163 annotation matching this pattern will be checked by this rule.
164- `match:label` - optional annotation filter, only rules with at least one label
165 matching this pattern will be checked by this rule. For recording rules only static
166 labels set on the recording rule are considered.
167- `ignore` - works exactly like `match` but does the opposite - any alerting or recording rule
168 matching all conditions defined on `ignore` will not be checked by this `rule` block.
169
170Example:
171
172```JS
173rule {
174 match {
175 path = "rules/.*"
176 kind = "alerting"
177 label "severity" {
178 value = "(warning|critical)"
179 }
180 [ check applied only to severity="critical" and severity="warning" alerts ]
181 }
182}
183```
184
185# Check definitions
186
187## Aggregation
188
189This check is used to inspect promql expressions and ensure that specific labels
190are kept or stripped away when aggregating results. It's mostly useful in recording
191rules.
192
193Syntax:
194
195```JS
196aggregate "(.*)" {
197 severity = "bug|warning|info"
198 keep = [ "...", ... ]
199 strip = [ "...", ... ]
200}
201```
202
203- `severity` - set custom severity for reported issues, defaults to a warning
204- `keep` - list of label names that must be preserved
205- `strip` - list of label names that must be stripped
206
207Examples:
208
209Ensure that all series generated from recording rules have `job` labels preserved:
210
211```JS
212rule {
213 match {
214 kind = "recording"
215 }
216 aggregate ".+" {
217 keep = ["job"]
218 }
219}
220```
221
222In some cases you might want to ensure that specific labels are removed in aggregations.
223For example in recording rules that are producing series consumed by federation, where
224only aggregated results (not per instance) are allowed:
225
226```JS
227rule {
228 match {
229 kind = "recording"
230 }
231 aggregate "cluster:.+" {
232 strip = ["instance"]
233 }
234}
235```
236
237By default all issues found by this check will be reported as warnings. To adjust
238severity set a custom `severity` key:
239
240```JS
241aggregate ".+" {
242 ...
243 severity = "bug"
244}
245```
246
247## Annotations
248
249This check is used to ensure that all required annotations are set on alerts and that
250they have correct values.
251
252Syntax:
253
254```JS
255annotation "(.*)" {
256 severity = "bug|warning|info"
257 value = "(.*)"
258 required = true|false
259}
260```
261
262- `severity` - set custom severity for reported issues, defaults to a warning
263- `value` - optional value pattern to enforce
264- `required` - if `true` pint will require every alert to have this annotation set,
265 if `false` it will only check values where annotation is set
266
267Examples:
268
269This set of rules will:
270- require `summary` annotation to be present, if missing it will be reported as a warning
271- if a `dashboard` annotation is provided it must match `https://grafana\.example\.com/.+`
272 pattern, if it doesn't match that pattern it will be reported as a bug
273
274```JS
275rule {
276 match {
277 kind = "alerting"
278 }
279
280 annotation "summary" {
281 required = true
282 }
283
284 annotation "dashboard" {
285 severity = "bug"
286 value = "https://grafana\.example\.com/.+"
287 }
288}
289```
290
291## Labels
292
293This check works the same way as `annotation` check, but it operates on
294labels instead.
295It uses static labels set on alerting or recording rule. It doesn't use
296labels on time series used in those rules.
297
298Syntax:
299
300```JS
301label "(.*)" {
302 severity = "bug|warning|info"
303 value = "..."
304 required = true|false
305}
306```
307
308Example:
309
310Require `severity` label to be set on alert rules with two all possible values:
311
312```JS
313rule {
314 match {
315 kind = "alerting"
316 }
317
318 label "severity" {
319 value = "(warning|critical)"
320 required = true
321 }
322}
323```
324
325## Rate
326
327This check inspects `rate()` and `irate()` functions and warns if used duration
328is too low. It does so by first getting global `scrape_interval` value for selected
329Prometheus servers and comparing duration to it.
330Reported issue depends on a few factors:
331
332For `rate()` function:
333- If duration is less than 2x `scrape_interval` it will report a bug.
334- If duration is between 2x and 4x `scrape_interval` it will report a warning.
335
336For `irate()` function:
337- If duration is less than 2x `scrape_interval` it will report a bug.
338- If duration is between 2x and 3x `scrape_interval` it will report a warning.
339
340Syntax:
341
342```JS
343rate {}
344```
345
346Example:
347
348```JS
349prometheus "prod" {
350 uri = "https://prometheus-prod.example.com"
351 timeout = "60s"
352}
353
354prometheus "dev" {
355 uri = "https://prometheus-dev.example.com"
356 timeout = "30s"
357}
358
359rule {
360 match {
361 kind = "recording"
362 }
363
364 rate {}
365}
366```
367
368## Alerts
369
370This check is used to estimate how many times given alert would fire.
371It will run `expr` query from every alert rule against selected Prometheus
372servers and report how many unique alerts it would generate.
373If `for` is set on alerts it will be used to adjust results.
374
375Syntax:
376
377```JS
378alerts {
379 range = "1h"
380 step = "1m"
381 resolve = "5m"
382}
383```
384
385- `range` - query range, how far to look back, `1h` would mean that pint will
386 query last 1h of metrics. If a query results in a timeout pint will retry it
387 with 50% smaller range until it succeeds.
388 Defaults to `1d`.
389- `step` - query resolution, for most accurate result use step equal
390 to `scrape_interval`, try to reduce it if that would load too many samples.
391 Defaults to `1m`.
392- `resolve` - duration after which stale alerts are resolved. Defaults to `5m`.
393
394Example:
395
396```JS
397prometheus "prod" {
398 uri = "https://prometheus-prod.example.com"
399 timeout = "60s"
400}
401
402rule {
403 match {
404 kind = "recording"
405 }
406 alerts {
407 range = "1d"
408 step = "1m"
409 resolve = "5m"
410 }
411}
412```
413
414## Comparison
415
416This check enforces use of a comparison operator in alert queries.
417Since any query result triggers an alert usual query would be something
418like `error_count > 10`, so we only get `error_count` series if the value
419is above 10. If we would remove `> 10` part query would always return `error_count`
420and so it would always trigger an alert.
421
422Syntax:
423
424```JS
425comparison {
426 severity = "bug|warning|info"
427}
428```
429
430- `severity` - set custom severity for reported issues, defaults to a bug.
431
432Example:
433
434```
435rule {
436 match {
437 kind = "alerting"
438 }
439 comparison {}
440}
441```
442
443
444## Cost
445
446This check is used to calculate cost of a query and optionally report an issue
447if that cost is too high. It will run `expr` query from every rule against
448selected Prometheus servers and report results.
449This check can be used for both recording and alerting rules, but is most
450useful for recording rules.
451
452Syntax:
453
454```JS
455cost {
456 severity = "bug|warning|info"
457 bytesPerSample = 1024
458 maxSeries = 5000
459}
460```
461
462- `severity` - set custom severity for reported issues, defaults to a warning.
463 This is only used when query result series exceed `maxSeries` value (if set).
464 If `maxSeries` is not set or when results count is below it pint will still
465 report it as information.
466- `bytesPerSample` - if set results will use this to calculate estimated memory
467 required to store returned series in Prometheus.
468- `maxSeries` - if set and number of results for given query exceeds this value
469 it will be reported as a bug (or custom severity if `severity` is set).
470
471Examples:
472
473All rules from files matching `rules/dev/.+` pattern will be tested against
474`dev` server. Results will be reported as information regardless of results.
475
476```JS
477prometheus "dev" {
478 uri = "https://prometheus-dev.example.com"
479 timeout = "30s"
480 paths = ["rules/dev/.+"]
481}
482
483rule {
484 cost {}
485}
486```
487
488To add memory usage estimate we first need to get average bytes per sample.
489This can be be estimated using two different queries:
490
491- for RSS usage: `process_resident_memory_bytes / prometheus_tsdb_head_series`
492- for Go allocations: `go_memstats_alloc_bytes / prometheus_tsdb_head_series`
493
494Since Go uses garbage collector RSS memory will be more than the sum of all
495memory allocations. RSS usage will be "worst case" while "Go alloc" best case,
496while real memory usage will be somewhere in between, depending on many factors
497like memory pressure, Go version, GOGC settings etc.
498
499```JS
500...
501 cost {
502 bytesPerSample = 4096
503 }
504}
505```
506
507## Series
508
509This check will also query Prometheus servers, it is used to warn about queries
510that are using metrics not currently present in Prometheus.
511It parses `expr` query from every rule, finds individual metric selectors and
512checks if they return any values.
513
514Let's say we have a rule this query: `sum(my_metric{foo="bar"}) > 10`.
515This checks would query all configured server for the existence of
516`my_metric{foo="bar"}` series and report a warning if it's missing.
517
518Syntax:
519
520```JS
521series {
522 severity = "bug|warning|info"
523}
524```
525
526- `severity` - set custom severity for reported issues, defaults to a warning.
527
528Example:
529
530```JS
531prometheus "dev" {
532 uri = "https://prometheus-dev.example.com"
533 timeout = "30s"
534}
535
536prometheus "prod" {
537 uri = "https://prometheus-prod.example.com"
538 timeout = "30s"
539}
540
541rule {
542 match {
543 kind = "recording"
544 }
545
546 series {}
547}
548```
549
550## Reject
551
552This check allows rejecting label or annotations keys and values
553using regexp rules.
554
555Syntax:
556
557```JS
558reject "(.*)" {
559 severity = "bug|warning|info"
560 label_keys = true|false
561 label_values = true|false
562 annotation_keys = true|false
563 annotation_values = true|false
564}
565```
566
567- `severity` - set custom severity for reported issues, defaults to a bug.
568- `label_keys` - if true label keys for recording and alerting rules will
569 be checked.
570- `label_values` - if true label values for recording and alerting rules will
571 be checked.
572- `annotation_keys` - if true annotation keys for alerting rules will be checked.
573- `annotation_values` - if true label values for alerting rules will be checked.
574
575Example:
576
577Disallow using URLs as label keys or values:
578
579```JS
580rule {
581 match {
582 kind = "alerting"
583 }
584
585 reject "https?://.+" {
586 label_keys = true
587 label_values = true
588 }
589}
590```
591
592Disallow spaces in label and annotation keys:
593
594```JS
595rule {
596 reject ".* +.*" {
597 annotation_keys = true
598 label_keys = true
599 }
600}
601```
602
603## Template
604
605This check validates templating used in annotations and labels for alerting rules.
606See [Prometheus docs](https://prometheus.io/docs/prometheus/latest/configuration/template_reference/)
607for details of supported templating syntax.
608
609This check will also inspect all alert rules and warn if any of them
610uses query return values inside alert labels.
611Two alerts are identical if they have identical labels, so using
612query value will generate a new unique alert every time it changes.
613If alerting rule is using `for` it might prevent it from ever firing
614if the value keeps changing before `for` is satisfied, because
615Prometheus will consider it to be a new alert and start `for` tracking
616from zero.
617
618If you want to include query value in the alert then use annotations
619for that. Annotations are not used to compare alerts identity and so
620the value of any annotation can change between alert evaluations.
621
622See [this blog post](https://www.robustperception.io/dont-put-the-value-in-alert-labels)
623for more details.
624
625Syntax:
626
627```JS
628template {
629 severity = "bug|warning|info"
630}
631```
632
633- `severity` - set custom severity for reported issues, defaults to a bug.
634
635Example:
636
637```JS
638rule {
639 match {
640 kind = "alerting"
641 }
642
643 template {
644 severity = "fatal"
645 }
646}
647```
648
649## Vector Matching
650
651This check will try to find queries that try to
652[match vectors](https://prometheus.io/docs/prometheus/latest/querying/operators/#vector-matching)
653but have different sets of labels on both side of the query.
654
655Consider these two time series:
656
657```
658http_errors{job="node-exporter", cluster="prod", instance="server1"}
659```
660
661and
662
663```
664cluster:http_errors{job="node-exporter", cluster="prod"}
665```
666
667One of them tracks specific instance and one aggregates series for the whole cluster.
668Because they have different set of labels if we want to calculate some value using both
669of them, for example:
670
671```
672http_errors / cluster:http_errors
673```
674
675we wouldn't get any results. To fix that we need ignore extra labels:
676
677```
678http_errors / ignoring(instance) cluster:http_errors
679```
680
681This check aims to find all queries that using vector matching where both sides
682of the query have different sets of labels causing no results to be returned.
683
684Syntax:
685
686```JS
687vector_matching {
688 severity = "bug|warning|info"
689}
690```
691
692- `severity` - set custom severity for reported issues, defaults to a warning.
693
694Example:
695
696```JS
697prometheus "dev" {
698 uri = "https://prometheus-dev.example.com"
699 timeout = "30s"
700}
701
702prometheus "prod" {
703 uri = "https://prometheus-prod.example.com"
704 timeout = "30s"
705}
706
707rule {
708 vector_matching {}
709}
710```
711
712# Ignoring selected lines or files
713
714While parsing files pint will look for special comment blocks and use them to
715exclude some parts all whole files from checks.
716
717## Ignoring whole files
718
719Add a `# pint ignore/file` comment on top of the file, everything below that line
720will be ignored.
721
722Example:
723
724```YAML
725# pint ignore/file
726
727groups:
728 - name: example
729 rules:
730 - record: job:http_inprogress_requests:sum
731 expr: sum by (job) (http_inprogress_requests)
732```
733
734## Ignoring individual lines
735
736To ignore just one line use `# pint ignore/line` at the end of that line or
737`# ignore/next-line` on the line before.
738This is useful if you're linting templates used to generate Prometheus
739configuration and it contains some extra lines that are not valid YAML.
740
741Example:
742
743```YAML
744{% set some_jinja_var1 = "bar" } # pint ignore/line
745groups:
746 - name: example
747 rules:
748 - record: job:http_inprogress_requests:sum
749 expr: sum by (job) (http_inprogress_requests)
750
751# pint ignore/next-line
752{% set some_jinja_var2 = "foo" }
753```
754
755## Ignoring a range of lines
756
757To ignore a part of a file wrap it with `# pint ignore/begin` and
758`# pint ignore/end` comments.
759
760Example:
761
762```YAML
763# pint ignore/begin
764{% set some_jinja_var1 = "bar" }
765{% set some_jinja_var2 = "foo" }
766# pint ignore/end
767
768groups:
769 - name: example
770 rules:
771 - record: job:http_inprogress_requests:sum
772 expr: sum by (job) (http_inprogress_requests)
773```
774
775## Disabling individual checks for specific rules
776
777To disable individual check for a specific rule use `# pint disable ...` comments.
778A single comment can only disable one check, so repeat it for every check you wish
779to disable.
780
781To disable `query/cost` check add `# pint disable query/cost` comment anywhere in
782the rule.
783
784Example:
785
786```YAML
787groups:
788 - name: example
789 rules:
790 - record: instance:http_requests_total:avg_over_time:1w
791 # pint disable query/cost
792 expr: avg_over_time(http_requests_total[1w]) by (instance)
793```
794
795```YAML
796groups:
797 - name: example
798 rules:
799 - record: instance:http_requests_total:avg_over_time:1w
800 expr: avg_over_time(http_requests_total[1w]) by (instance) # pint disable query/cost
801```
802