safedep/vet
1
0
Fork 0
mirror of https://github.com/safedep/vet.git synced 2025-12-10 00:22:08 -06:00
Code Issues Packages Projects Releases 68 Wiki Activity

Refactor to support customer experimental parsers

Browse Source 
Add python wheel dist reading support

Run go mod tidy

Update README docs

Add justification for exceptions workflow

Misc fix for markdown reporting template
This commit is contained in:
abhisek 2023-03-16 16:16:22 +05:30
parent ab3d7e9f34
commit 9e1cc18faa
No known key found for this signature in database
GPG Key ID: CB92A4990C02A88F
12 changed files with 242 additions and 93 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

90
README.md
View File

@ -1,4 +1,5 @@
# vet
## Automate Open Source Package Vetting in CI/CD
`vet` is a tool for identifying risks in open source software supply chain. It
helps engineering and security teams to identify potential issues in their open
@ -10,7 +11,7 @@ source dependencies and evaluate them against organizational policies.
## Demo
[![asciicast](https://asciinema.org/a/I60aD2VtVsETQtIFsYTCewJZ3.svg)](https://asciinema.org/a/I60aD2VtVsETQtIFsYTCewJZ3)
![vet Demo](docs/images/vet-demo.gif)
## TL;DR
@ -40,13 +41,13 @@ Alternatively, look at [Releases](https://github.com/safedep/vet/releases) for
a pre-built binary for your platform. [SLSA Provenance](https://slsa.dev/provenance/v0.1) is published
along with each binary release.
Get a trial API key for [Insights API](https://safedep.io/docs/concepts/raya-data-platform-overview) access
Get an API key for [Insights API](https://safedep.io/docs/concepts/raya-data-platform-overview) access
```bash
vet auth trial --email john.doe@example.com
```
> A time limited trial API key will be sent over email.
> A time limited API key will be sent over email.
Configure `vet` to use API Key to access [Insights API](https://safedep.io/docs/concepts/raya-data-platform-overview)
@ -54,7 +55,7 @@ Configure `vet` to use API Key to access [Insights API](https://safedep.io/docs/
vet auth configure
```
> Insights API is used to enrich OSS packages with meta-data for rich query and policy
> Insights API is used to enrich OSS packages with metadata for rich query and policy
> decisions. Alternatively, the API key can be passed through environment
> variable `VET_API_KEY`
@ -90,7 +91,59 @@ The default scan uses an opinionated [Summary Reporter](#) which presents
a consolidated summary of findings. Thats NOT about it. Read more for
expression based filtering and policy evaluation.
## Filtering
## Policy Control
Policies are written using a DSL. A group of policies can be applied using
`vet` to build a security gate in CI/CD.
Start by copying a sample policy
```bash
curl -LO https://raw.githubusercontent.com/safedep/vet/main/samples/filter-suites/fs-generic.yml
```
Run a scan with policies and configure the scanner to fail in case of policy
violation
```bash
vet scan -D /path/to/dir --filter-suite fs-generic.yml --filter-fail
```
Read more about underlying capability using which policy control is implemented
in [filtering guide](docs/filtering.md)
## Exceptions Management
Projects may have legacy libraries that will fail any reasonable security policy.
Legacy libraries can be added as time bounded exceptions to the policies to place
strict control on any new library while legacy library can be upgraded over
time.
Exception rules can be generated using the `query` workflow to temporarily
ignore (or snooze) existing issues when using `vet` for the first time. This
helps in establishing security gating to prevent introduction of new security
issues while existing issues are being remediated.
Use exception rules during scan to ignore specific packages
```bash
vet scan -D /path/to/repo -e /path/to/exceptions.yml
```
For more information on generating exceptions,
refer to [exceptions guide](docs/exceptions.md)
The generated exceptions file, when combined with policy control, can be used
to setup a security gate to prevent introducing new issues while ignoring the
existing backlog for a period of time.
```bash
vet scan -D /path/to/dir \
--filter-suite fs-generic.yml --filter-fail
-e /path/to/exceptions.yml
```
## Exploring OSS Risks using Filters
Find dependencies that seems not very popular
@ -109,37 +162,10 @@ vet scan --lockfiles /path/to/pom.xml --report-summary=false \
> Use filtering along with `query` command for offline slicing and dicing of
> enriched package manifests. Read [filtering guide](docs/filtering.md)
Learn more about [filtering with vet](docs/filtering.md).
Look at [filter input spec](api/filter_input_spec.proto) on attributes
available to the filter expression.
### Using Filter Suite
Filter suites can be used to implement security gating in CI. [Example](samples/filter-suites/fs-generic.yml)
file suite contains rules to enforce generic OSS consumption best practices.
```bash
vet scan -D /path/to/dir --filter-suite /path/to/suite.yml --filter-fail
```
Read more about filter suites in [filtering guide](docs/filtering.md)
## Exceptions Management
Exception rules can be generated using the `query` workflow to temporarily
ignore (or snooze) existing issues when using `vet` for the first time. This
helps in establishing security gating to prevent introduction of new security
issues while existing issues are being remediated.
Use exception rules during scan to ignore specific packages
```bash
vet scan -D /path/to/repo -e /path/to/exceptions.yml
```
For more information, refer to [exceptions guide](docs/exceptions.md)
## FAQ
### How do I disable the stupid banner?

1
api/filter_input_spec.proto
View File

@ -20,6 +20,7 @@ message Vulnerabilities {
// OpenSSF Scorecard
message Scorecard {
map<string, float> scores = 1;
float score = 2;
}
enum ProjectType {

16
docs/filtering.md
View File

@ -4,7 +4,7 @@ Filter command helps solve the problem of visibility for OSS dependencies in an
application. To support various requirements, we adopt a generic [expressions
language](https://github.com/google/cel-spec) for flexible filtering.
Example:
## Example
```bash
vet scan -D /path/to/repo \
@ -14,6 +14,20 @@ vet scan -D /path/to/repo \
The scan will list only packages that use the `MIT` license.
Find dependencies that seems not very popular
```bash
vet scan --lockfiles /path/to/pom.xml --report-summary=false \
--filter='projects.exists(x, x.stars < 10)'
```
Find dependencies with a critical vulnerability
```bash
vet scan --lockfiles /path/to/pom.xml --report-summary=false \
--filter='vulns.critical.exists_one(x, true)'
```
## Input
Filter expressions work on packages (aka. dependencies) and evaluates to

BIN
docs/images/vet-demo.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.6 MiB

93
gen/filterinput/filter_input_spec.pb.go
View File

@ -208,6 +208,7 @@ type Scorecard struct {
unknownFields protoimpl.UnknownFields
Scores map[string]float32 `protobuf:"bytes,1,rep,name=scores,proto3" json:"scores,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"fixed32,2,opt,name=value,proto3"`
Score float32 `protobuf:"fixed32,2,opt,name=score,proto3" json:"score,omitempty"`
}
func (x *Scorecard) Reset() {
@ -249,6 +250,13 @@ func (x *Scorecard) GetScores() map[string]float32 {
return nil
}
func (x *Scorecard) GetScore() float32 {
if x != nil {
return x.Score
}
return 0
}
type ProjectInfo struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@ -490,48 +498,49 @@ var file_filter_input_spec_proto_rawDesc = []byte{
0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x56, 0x75, 0x6c, 0x6e, 0x65, 0x72, 0x61, 0x62, 0x69, 0x6c, 0x69,
0x74, 0x79, 0x52, 0x06, 0x6d, 0x65, 0x64, 0x69, 0x75, 0x6d, 0x12, 0x20, 0x0a, 0x03, 0x6c, 0x6f,
0x77, 0x18, 0x05, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x56, 0x75, 0x6c, 0x6e, 0x65, 0x72,
0x61, 0x62, 0x69, 0x6c, 0x69, 0x74, 0x79, 0x52, 0x03, 0x6c, 0x6f, 0x77, 0x22, 0x76, 0x0a, 0x09,
0x53, 0x63, 0x6f, 0x72, 0x65, 0x63, 0x61, 0x72, 0x64, 0x12, 0x2e, 0x0a, 0x06, 0x73, 0x63, 0x6f,
0x72, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x16, 0x2e, 0x53, 0x63, 0x6f, 0x72,
0x65, 0x63, 0x61, 0x72, 0x64, 0x2e, 0x53, 0x63, 0x6f, 0x72, 0x65, 0x73, 0x45, 0x6e, 0x74, 0x72,
0x79, 0x52, 0x06, 0x73, 0x63, 0x6f, 0x72, 0x65, 0x73, 0x1a, 0x39, 0x0a, 0x0b, 0x53, 0x63, 0x6f,
0x72, 0x65, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18,
0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61,
0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x02, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65,
0x3a, 0x02, 0x38, 0x01, 0x22, 0x87, 0x01, 0x0a, 0x0b, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74,
0x49, 0x6e, 0x66, 0x6f, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01,
0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x20, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65,
0x18, 0x02, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x0c, 0x2e, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74,
0x54, 0x79, 0x70, 0x65, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x14, 0x0a, 0x05, 0x73, 0x74,
0x61, 0x72, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x05, 0x73, 0x74, 0x61, 0x72, 0x73,
0x12, 0x14, 0x0a, 0x05, 0x66, 0x6f, 0x72, 0x6b, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x05, 0x52,
0x05, 0x66, 0x6f, 0x72, 0x6b, 0x73, 0x12, 0x16, 0x0a, 0x06, 0x69, 0x73, 0x73, 0x75, 0x65, 0x73,
0x18, 0x05, 0x20, 0x01, 0x28, 0x05, 0x52, 0x06, 0x69, 0x73, 0x73, 0x75, 0x65, 0x73, 0x22, 0x5c,
0x0a, 0x0e, 0x50, 0x61, 0x63, 0x6b, 0x61, 0x67, 0x65, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e,
0x12, 0x1c, 0x0a, 0x09, 0x65, 0x63, 0x6f, 0x73, 0x79, 0x73, 0x74, 0x65, 0x6d, 0x18, 0x01, 0x20,
0x01, 0x28, 0x09, 0x52, 0x09, 0x65, 0x63, 0x6f, 0x73, 0x79, 0x73, 0x74, 0x65, 0x6d, 0x12, 0x12,
0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61,
0x6d, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x18, 0x03, 0x20,
0x01, 0x28, 0x09, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x22, 0xc8, 0x01, 0x0a,
0x0b, 0x46, 0x69, 0x6c, 0x74, 0x65, 0x72, 0x49, 0x6e, 0x70, 0x75, 0x74, 0x12, 0x21, 0x0a, 0x03,
0x70, 0x6b, 0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0f, 0x2e, 0x50, 0x61, 0x63, 0x6b,
0x61, 0x67, 0x65, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x52, 0x03, 0x70, 0x6b, 0x67, 0x12,
0x26, 0x0a, 0x05, 0x76, 0x75, 0x6c, 0x6e, 0x73, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10,
0x2e, 0x56, 0x75, 0x6c, 0x6e, 0x65, 0x72, 0x61, 0x62, 0x69, 0x6c, 0x69, 0x74, 0x69, 0x65, 0x73,
0x52, 0x05, 0x76, 0x75, 0x6c, 0x6e, 0x73, 0x12, 0x28, 0x0a, 0x09, 0x73, 0x63, 0x6f, 0x72, 0x65,
0x63, 0x61, 0x72, 0x64, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0a, 0x2e, 0x53, 0x63, 0x6f,
0x72, 0x65, 0x63, 0x61, 0x72, 0x64, 0x52, 0x09, 0x73, 0x63, 0x6f, 0x72, 0x65, 0x63, 0x61, 0x72,
0x64, 0x12, 0x28, 0x0a, 0x08, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x18, 0x04, 0x20,
0x03, 0x28, 0x0b, 0x32, 0x0c, 0x2e, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x49, 0x6e, 0x66,
0x6f, 0x52, 0x08, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x12, 0x1a, 0x0a, 0x08, 0x6c,
0x69, 0x63, 0x65, 0x6e, 0x73, 0x65, 0x73, 0x18, 0x05, 0x20, 0x03, 0x28, 0x09, 0x52, 0x08, 0x6c,
0x69, 0x63, 0x65, 0x6e, 0x73, 0x65, 0x73, 0x2a, 0x26, 0x0a, 0x0b, 0x50, 0x72, 0x6f, 0x6a, 0x65,
0x63, 0x74, 0x54, 0x79, 0x70, 0x65, 0x12, 0x0b, 0x0a, 0x07, 0x55, 0x4e, 0x4b, 0x4e, 0x4f, 0x57,
0x4e, 0x10, 0x00, 0x12, 0x0a, 0x0a, 0x06, 0x47, 0x49, 0x54, 0x48, 0x55, 0x42, 0x10, 0x01, 0x42,
0x28, 0x5a, 0x26, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x73, 0x61,
0x66, 0x65, 0x64, 0x65, 0x70, 0x2f, 0x76, 0x65, 0x74, 0x2f, 0x67, 0x65, 0x6e, 0x2f, 0x66, 0x69,
0x6c, 0x74, 0x65, 0x72, 0x69, 0x6e, 0x70, 0x75, 0x74, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x33,
0x61, 0x62, 0x69, 0x6c, 0x69, 0x74, 0x79, 0x52, 0x03, 0x6c, 0x6f, 0x77, 0x22, 0x8c, 0x01, 0x0a,
0x09, 0x53, 0x63, 0x6f, 0x72, 0x65, 0x63, 0x61, 0x72, 0x64, 0x12, 0x2e, 0x0a, 0x06, 0x73, 0x63,
0x6f, 0x72, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x16, 0x2e, 0x53, 0x63, 0x6f,
0x72, 0x65, 0x63, 0x61, 0x72, 0x64, 0x2e, 0x53, 0x63, 0x6f, 0x72, 0x65, 0x73, 0x45, 0x6e, 0x74,
0x72, 0x79, 0x52, 0x06, 0x73, 0x63, 0x6f, 0x72, 0x65, 0x73, 0x12, 0x14, 0x0a, 0x05, 0x73, 0x63,
0x6f, 0x72, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x02, 0x52, 0x05, 0x73, 0x63, 0x6f, 0x72, 0x65,
0x1a, 0x39, 0x0a, 0x0b, 0x53, 0x63, 0x6f, 0x72, 0x65, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12,
0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65,
0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x02,
0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22, 0x87, 0x01, 0x0a, 0x0b,
0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x49, 0x6e, 0x66, 0x6f, 0x12, 0x12, 0x0a, 0x04, 0x6e,
0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12,
0x20, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x0c, 0x2e,
0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x54, 0x79, 0x70, 0x65, 0x52, 0x04, 0x74, 0x79, 0x70,
0x65, 0x12, 0x14, 0x0a, 0x05, 0x73, 0x74, 0x61, 0x72, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05,
0x52, 0x05, 0x73, 0x74, 0x61, 0x72, 0x73, 0x12, 0x14, 0x0a, 0x05, 0x66, 0x6f, 0x72, 0x6b, 0x73,
0x18, 0x04, 0x20, 0x01, 0x28, 0x05, 0x52, 0x05, 0x66, 0x6f, 0x72, 0x6b, 0x73, 0x12, 0x16, 0x0a,
0x06, 0x69, 0x73, 0x73, 0x75, 0x65, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x05, 0x52, 0x06, 0x69,
0x73, 0x73, 0x75, 0x65, 0x73, 0x22, 0x5c, 0x0a, 0x0e, 0x50, 0x61, 0x63, 0x6b, 0x61, 0x67, 0x65,
0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x12, 0x1c, 0x0a, 0x09, 0x65, 0x63, 0x6f, 0x73, 0x79,
0x73, 0x74, 0x65, 0x6d, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x65, 0x63, 0x6f, 0x73,
0x79, 0x73, 0x74, 0x65, 0x6d, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20,
0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72,
0x73, 0x69, 0x6f, 0x6e, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73,
0x69, 0x6f, 0x6e, 0x22, 0xc8, 0x01, 0x0a, 0x0b, 0x46, 0x69, 0x6c, 0x74, 0x65, 0x72, 0x49, 0x6e,
0x70, 0x75, 0x74, 0x12, 0x21, 0x0a, 0x03, 0x70, 0x6b, 0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b,
0x32, 0x0f, 0x2e, 0x50, 0x61, 0x63, 0x6b, 0x61, 0x67, 0x65, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f,
0x6e, 0x52, 0x03, 0x70, 0x6b, 0x67, 0x12, 0x26, 0x0a, 0x05, 0x76, 0x75, 0x6c, 0x6e, 0x73, 0x18,
0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x56, 0x75, 0x6c, 0x6e, 0x65, 0x72, 0x61, 0x62,
0x69, 0x6c, 0x69, 0x74, 0x69, 0x65, 0x73, 0x52, 0x05, 0x76, 0x75, 0x6c, 0x6e, 0x73, 0x12, 0x28,
0x0a, 0x09, 0x73, 0x63, 0x6f, 0x72, 0x65, 0x63, 0x61, 0x72, 0x64, 0x18, 0x03, 0x20, 0x01, 0x28,
0x0b, 0x32, 0x0a, 0x2e, 0x53, 0x63, 0x6f, 0x72, 0x65, 0x63, 0x61, 0x72, 0x64, 0x52, 0x09, 0x73,
0x63, 0x6f, 0x72, 0x65, 0x63, 0x61, 0x72, 0x64, 0x12, 0x28, 0x0a, 0x08, 0x70, 0x72, 0x6f, 0x6a,
0x65, 0x63, 0x74, 0x73, 0x18, 0x04, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0c, 0x2e, 0x50, 0x72, 0x6f,
0x6a, 0x65, 0x63, 0x74, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x08, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63,
0x74, 0x73, 0x12, 0x1a, 0x0a, 0x08, 0x6c, 0x69, 0x63, 0x65, 0x6e, 0x73, 0x65, 0x73, 0x18, 0x05,
0x20, 0x03, 0x28, 0x09, 0x52, 0x08, 0x6c, 0x69, 0x63, 0x65, 0x6e, 0x73, 0x65, 0x73, 0x2a, 0x26,
0x0a, 0x0b, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x54, 0x79, 0x70, 0x65, 0x12, 0x0b, 0x0a,
0x07, 0x55, 0x4e, 0x4b, 0x4e, 0x4f, 0x57, 0x4e, 0x10, 0x00, 0x12, 0x0a, 0x0a, 0x06, 0x47, 0x49,
0x54, 0x48, 0x55, 0x42, 0x10, 0x01, 0x42, 0x28, 0x5a, 0x26, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62,
0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x73, 0x61, 0x66, 0x65, 0x64, 0x65, 0x70, 0x2f, 0x76, 0x65, 0x74,
0x2f, 0x67, 0x65, 0x6e, 0x2f, 0x66, 0x69, 0x6c, 0x74, 0x65, 0x72, 0x69, 0x6e, 0x70, 0x75, 0x74,
0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (

8
go.mod
View File

@ -6,13 +6,13 @@ require (
github.com/deepmap/oapi-codegen v1.12.4
github.com/golang/protobuf v1.5.2
github.com/google/cel-go v0.13.0
github.com/google/osv-scanner v1.1.0
github.com/google/osv-scanner v1.2.0
github.com/jedib0t/go-pretty/v6 v6.4.4
github.com/safedep/dry v0.0.0-20230222132026-c8b6cb976849
github.com/sirupsen/logrus v1.9.0
github.com/spf13/cobra v1.6.1
github.com/stretchr/testify v1.8.1
golang.org/x/term v0.4.0
golang.org/x/term v0.5.0
google.golang.org/protobuf v1.28.1
gopkg.in/yaml.v2 v2.4.0
)
@ -32,8 +32,8 @@ require (
github.com/rivo/uniseg v0.2.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stoewer/go-strcase v1.2.0 // indirect
golang.org/x/mod v0.7.0 // indirect
golang.org/x/sys v0.4.0 // indirect
golang.org/x/mod v0.8.0 // indirect
golang.org/x/sys v0.5.0 // indirect
golang.org/x/text v0.5.0 // indirect
google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect

18
go.sum
View File

@ -24,8 +24,8 @@ github.com/google/cel-go v0.13.0 h1:z+8OBOcmh7IeKyqwT/6IlnMvy621fYUqnTVPEdegGlU=
github.com/google/cel-go v0.13.0/go.mod h1:K2hpQgEjDp18J76a2DKFRlPBPpgRZgi6EbnpDgIhJ8s=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/osv-scanner v1.1.0 h1:6XL8tD8u4w8NFyiMo03Yd4xGG1VXhZXyrBESBuyWeUY=
github.com/google/osv-scanner v1.1.0/go.mod h1:w8BdEP4PJSosGhDfZ6W5RGMfIGb73rW38vCXB9DWA4c=
github.com/google/osv-scanner v1.2.0 h1:TWPfI5kDqO/wgxihVyRxX15JQFJeZ0NyPvx1UtR6g38=
github.com/google/osv-scanner v1.2.0/go.mod h1:2GmR3DxMxDW/HkkgOf+b/KCfspX2Ls4wcapeNnSNZoY=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc=
@ -46,8 +46,6 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55 h1:OBzggSWzjyEa7YaXp2DvpKDe1wYXtOEcFXQfDqkK7PI=
github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55/go.mod h1:odFOtG1l46k23IaCY6kdNkkLW8L+NT+EUVYYVphP59I=
github.com/safedep/dry v0.0.0-20230222132026-c8b6cb976849 h1:5nO9ht1jn7XHFyNFRhUneDZbKmwh4kRr0w/EoWuOQQA=
github.com/safedep/dry v0.0.0-20230222132026-c8b6cb976849/go.mod h1:odFOtG1l46k23IaCY6kdNkkLW8L+NT+EUVYYVphP59I=
github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0=
@ -70,14 +68,14 @@ github.com/stretchr/testify v1.7.4/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
golang.org/x/mod v0.7.0 h1:LapD9S96VoQRhi/GrNTqeBJFrUjs5UHCAtTlgwA5oZA=
golang.org/x/mod v0.7.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.8.0 h1:LUYupSeNrTNCGzR/hVBk2NHZO4hXcVaW1k4Qx7rjPx8=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.4.0 h1:Zr2JFtRQNX3BCZ8YtxRE9hNJYC8J6I1MVbMg6owUp18=
golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.4.0 h1:O7UWfv5+A2qiuulQk30kVinPoMtoIPeVaKLEgLpVkvg=
golang.org/x/term v0.4.0/go.mod h1:9P2UbLfCdcvo3p/nzKvsmas4TnlujnuoV9hGgYzW1lQ=
golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/text v0.5.0 h1:OLmvp0KP+FVG99Ct/qFiL/Fhk4zp4QQnZ7b2U+5piUM=
golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

7
pkg/analyzer/filter/eval.go
View File

@ -251,7 +251,12 @@ func (f *filterEvaluator) buildFilterInput(pkg *models.Package) (*filterinput.Fi
// Scorecard
scorecard := utils.SafelyGetValue(insight.Scorecard)
checks := utils.SafelyGetValue(utils.SafelyGetValue(scorecard.Content).Checks)
scorecardContent := utils.SafelyGetValue(scorecard.Content)
// Aggregated score
fi.Scorecard.Score = utils.SafelyGetValue(scorecardContent.Score)
checks := utils.SafelyGetValue(scorecardContent.Checks)
for _, check := range checks {
fi.Scorecard.Scores[string(utils.SafelyGetValue(check.Name))] =
utils.SafelyGetValue(check.Score)

17
pkg/parser/parser.go
View File

@ -8,6 +8,10 @@ import (
"github.com/safedep/vet/pkg/models"
)
const (
customParserTypePyWheel = "python-wheel"
)
// We are supporting only those ecosystems for which we have data
// for enrichment. More ecosystems will be supported as we improve
// the capability of our Insights API
@ -18,6 +22,10 @@ var supportedEcosystems map[string]bool = map[string]bool{
models.EcosystemPyPI: true,
}
var customExperimentalParsers map[string]lockfile.PackageDetailsParser = map[string]lockfile.PackageDetailsParser{
customParserTypePyWheel: parsePythonWheelDist,
}
type Parser interface {
Ecosystem() string
Parse(lockfilePath string) (models.PackageManifest, error)
@ -53,6 +61,13 @@ func FindParser(lockfilePath, lockfileAs string) (Parser, error) {
}
}
if p, ok := customExperimentalParsers[lockfileAs]; ok {
pw := &parserWrapper{parser: p, parseAs: lockfileAs}
if pw.supported() {
return pw, nil
}
}
return nil, fmt.Errorf("no parser found with: %s for: %s", lockfileAs,
lockfilePath)
}
@ -93,6 +108,8 @@ func (pw *parserWrapper) Ecosystem() string {
return models.EcosystemMaven
case "buildscript-gradle.lockfile":
return models.EcosystemMaven
case customParserTypePyWheel:
return models.EcosystemPyPI
default:
return ""
}

79
pkg/parser/pywheel.go Normal file
View File

@ -0,0 +1,79 @@
package parser
import (
"archive/zip"
"errors"
"io"
"net/mail"
"strings"
"github.com/google/osv-scanner/pkg/lockfile"
"github.com/safedep/vet/pkg/common/logger"
)
// https://packaging.python.org/en/latest/specifications/binary-distribution-format/
func parsePythonWheelDist(pathToLockfile string) ([]lockfile.PackageDetails, error) {
details := []lockfile.PackageDetails{}
r, err := zip.OpenReader(pathToLockfile)
if err != nil {
return details, err
}
defer r.Close()
for _, file := range r.File {
if strings.HasSuffix(file.Name, ".dist-info/METADATA") {
fd, err := file.Open()
if err != nil {
return details, err
}
defer fd.Close()
return parsePythonPkgInfo(fd)
}
}
return details, errors.New("no METADATA found inside wheel")
}
// https://packaging.python.org/en/latest/specifications/core-metadata/
func parsePythonPkgInfo(reader io.Reader) ([]lockfile.PackageDetails, error) {
m, err := mail.ReadMessage(reader)
if err != nil {
return []lockfile.PackageDetails{}, err
}
// https://packaging.python.org/en/latest/specifications/core-metadata/#requires-dist-multiple-use
if dists, ok := m.Header["Requires-Dist"]; ok {
details := []lockfile.PackageDetails{}
for _, dist := range dists {
p, err := parsePythonPackageSpec(dist)
if err != nil {
logger.Errorf("Failed to parse python pkg spec: %s err: %v",
dist, err)
continue
}
details = append(details, p)
}
return details, nil
}
return []lockfile.PackageDetails{}, nil
}
// https://peps.python.org/pep-0440/
// https://peps.python.org/pep-0508/
// Parsing python dist version spec is not easy. We need to use the spec grammar
// to do it correctly. Taking shortcut here by only using the name as the first
// iteration ignoring the version
func parsePythonPackageSpec(pkgSpec string) (lockfile.PackageDetails, error) {
name := strings.SplitN(pkgSpec, " ", 2)[0]
return lockfile.PackageDetails{
Name: name,
Version: "0.0.0",
Ecosystem: lockfile.PipEcosystem,
CompareAs: lockfile.PipEcosystem,
}, nil
}

4
pkg/reporter/markdown.template.md
View File

@ -23,8 +23,8 @@ issues identified during the scan.
{{ range $key, $value := .Remediations }}
> {{ $key }}
| Package | Update Version | Risk Score | Issues |
|---------|----------------|------------|--------|
| Package | Update Version | Impact Score | Issues |
|---------|----------------|--------------|--------|
{{- range $value }}
| {{ .PkgRemediationName }} | {{ .Pkg.Insights.PackageCurrentVersion }} | {{ .Score }} | - |
{{- end }}

2
pkg/scanner/scanner.go
View File

@ -56,7 +56,7 @@ func (s *packageManifestScanner) ScanDirectory(dir string) error {
// automatic parser selection
func (s *packageManifestScanner) ScanLockfiles(lockfiles []string,
lockfileAs string) error {
logger.Infof("Scannding %d lockfiles as %s", len(lockfiles), lockfileAs)
logger.Infof("Scanning %d lockfiles as %s", len(lockfiles), lockfileAs)
manifests, err := scanLockfilesForManifests(lockfiles, lockfileAs)
if err != nil {