mirror of
https://github.com/safedep/vet.git
synced 2025-12-10 00:22:08 -06:00
Merge pull request #25 from safedep/develop
Add Support for Package Exceptions
This commit is contained in:
commit
56fdeb64ce
11
Makefile
11
Makefile
@ -25,9 +25,18 @@ protoc-codegen:
|
||||
--go_out=./gen/filtersuite \
|
||||
--go_opt=paths=source_relative \
|
||||
./api/filter_suite_spec.proto
|
||||
protoc -I ./api \
|
||||
--go_out=./gen/exceptionsapi \
|
||||
--go_opt=paths=source_relative \
|
||||
./api/exceptions_spec.proto
|
||||
|
||||
setup:
|
||||
mkdir -p out gen/insightapi gen/controlplane gen/filterinput gen/filtersuite
|
||||
mkdir -p out \
|
||||
gen/insightapi \
|
||||
gen/controlplane \
|
||||
gen/filterinput \
|
||||
gen/filtersuite \
|
||||
gen/exceptionsapi
|
||||
|
||||
GO_CFLAGS=-X main.commit=$(GITCOMMIT) -X main.version=$(VERSION)
|
||||
GO_LDFLAGS=-ldflags "-w $(GO_CFLAGS)"
|
||||
|
||||
32
README.md
32
README.md
@ -14,6 +14,17 @@ source dependencies and evaluate them against organizational policies.
|
||||
|
||||
## TL;DR
|
||||
|
||||
Scan a repository for OSS dependency risks with auto-detection of package
|
||||
manifests
|
||||
|
||||
```bash
|
||||
vet scan -D /path/to/repo
|
||||
```
|
||||
|
||||

|
||||
|
||||
## Getting Started
|
||||
|
||||
> Ensure `$(go env GOPATH)/bin` is in your `$PATH`
|
||||
|
||||
Install using `go get`
|
||||
@ -57,6 +68,12 @@ vet scan --lockfiles /path/to/requirements.txt
|
||||
vet scan --lockfiles /path/to/package-lock.json
|
||||
```
|
||||
|
||||
or scan a supported package manifest with a non-standard name
|
||||
|
||||
```bash
|
||||
vet scan --lockfiles /path/to/gradle-compileOnly.lock --lockfile-as gradle.lockfile
|
||||
```
|
||||
|
||||
> Use `vet scan parsers` to list supported package manifest parsers
|
||||
|
||||
The default scan uses an opinionated [Summary Reporter](#) which presents
|
||||
@ -98,6 +115,21 @@ vet scan -D /path/to/dir --filter-suite /path/to/suite.yml --filter-fail
|
||||
|
||||
Read more about filter suites in [filtering guide](docs/filtering.md)
|
||||
|
||||
## Exceptions Management
|
||||
|
||||
Exception rules can be generated using the `query` workflow to temporarily
|
||||
ignore (or snooze) existing issues when using `vet` for the first time. This
|
||||
helps in establishing security gating to prevent introduction of new security
|
||||
issues while existing issues are being remediated.
|
||||
|
||||
Use exception rules during scan to ignore specific packages
|
||||
|
||||
```bash
|
||||
vet scan -D /path/to/repo -e /path/to/exceptions.yml
|
||||
```
|
||||
|
||||
For more information, refer to [exceptions guide](docs/exceptions.md)
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I disable the stupid banner?
|
||||
|
||||
18
api/exceptions_spec.proto
Normal file
18
api/exceptions_spec.proto
Normal file
@ -0,0 +1,18 @@
|
||||
syntax = "proto3";
|
||||
|
||||
option go_package = "github.com/safedep/vet/gen/exceptionsapi";
|
||||
|
||||
message Exception {
|
||||
string id = 1;
|
||||
string ecosystem = 2;
|
||||
string name = 3;
|
||||
string version = 4;
|
||||
string expires = 5;
|
||||
string pattern = 6; // To be used for special cases
|
||||
}
|
||||
|
||||
message ExceptionSuite {
|
||||
string name = 1;
|
||||
string description = 2;
|
||||
repeated Exception exceptions = 3;
|
||||
}
|
||||
84
docs/exceptions.md
Normal file
84
docs/exceptions.md
Normal file
@ -0,0 +1,84 @@
|
||||
# Exceptions Management
|
||||
|
||||
Any security scanning tool may produce
|
||||
|
||||
1. False positive
|
||||
2. Issues that are acceptable for a period of time
|
||||
3. Issues that are ignored permanently
|
||||
|
||||
To support exceptions, we introduce the exception model defined in [exception
|
||||
spec](../api/exceptions_spec.proto)
|
||||
|
||||
## Use-case
|
||||
|
||||
> As a user of `vet` tool, I want to add all existing packages or package
|
||||
> versions as `exceptions` to make the scanner and filter analyser to ignore
|
||||
> them while reporting issues so that I can deploy `vet` as a security gate to
|
||||
> prevent introducing new packages with security issues
|
||||
|
||||
This workflow will allow users to
|
||||
|
||||
1. Accept the current issues as backlog to be mitigated over time
|
||||
2. Deploy `vet` as a security gate in CI to prevent introducing new issues
|
||||
|
||||
### Security Risks
|
||||
|
||||
Exceptions management should handle the potential security risk of ignoring a
|
||||
package and its future issues. To mitigate this risk, we will ensure that issues
|
||||
can be ignored till an acceptable time window and not permanently.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Generate Exceptions File
|
||||
|
||||
Run a scan and dump raw data to a directory
|
||||
|
||||
```bash
|
||||
vet scan -D /path/to/repo --json-dump-dir /path/to/dump
|
||||
```
|
||||
|
||||
Use `vet query` to generate exceptions for all existing packages
|
||||
|
||||
```bash
|
||||
vet query --from /path/to/dump \
|
||||
--exceptions-generate /path/to/exceptions.yml \
|
||||
--exceptions-filter 'true' \ # Optional filter for packages to add
|
||||
--exceptions-till '2023-12-12'
|
||||
```
|
||||
|
||||
> `--exceptions-till` is parsed as `YYYY-mm-dd` and will generate a timestamp
|
||||
> of `00:00:00` in UTC timezone for the date in RFC3339 format
|
||||
|
||||
### Customize Exceptions File
|
||||
|
||||
The generated exceptions file will add all packages, matching optional filter,
|
||||
into the `exceptions.yml` file. This file should be reviewed and customised as
|
||||
required before using it.
|
||||
|
||||
### Use Exceptions to Ignore Specific Packages
|
||||
|
||||
An exceptions file can be passed as a global flag to `vet`. It will be used for
|
||||
various commands such as `scan` or `query`.
|
||||
|
||||
```bash
|
||||
./vet --exceptions /path/to/exceptions.yml scan -D /path/to/repo
|
||||
```
|
||||
|
||||
> **Note:** Do not pass this flag while generating exceptions list in query
|
||||
> workflow to avoid incorrect exception list generation
|
||||
|
||||
## Behaviour
|
||||
|
||||
* All exceptions rules are applied only on a `Package`
|
||||
* All comparisons will be case-insensitive except version
|
||||
* Only `version` can have a value of `*` matching any version
|
||||
* Exceptions are globally managed and will be shared across packages
|
||||
* Exempted packages will be ignored by all analysers and reporters
|
||||
* First match policy for exceptions matching
|
||||
|
||||
Anti-patterns that will NOT be implemented
|
||||
|
||||
* Exceptions will not be implemented for manifests because they will cause
|
||||
false negatives
|
||||
* Exceptions will not be created without an expiry to avoid future false
|
||||
negatives on the package
|
||||
BIN
docs/images/vet-summary-demo.png
Normal file
BIN
docs/images/vet-summary-demo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 82 KiB |
275
gen/exceptionsapi/exceptions_spec.pb.go
Normal file
275
gen/exceptionsapi/exceptions_spec.pb.go
Normal file
@ -0,0 +1,275 @@
|
||||
// Code generated by protoc-gen-go. DO NOT EDIT.
|
||||
// versions:
|
||||
// protoc-gen-go v1.28.0
|
||||
// protoc v3.18.0
|
||||
// source: exceptions_spec.proto
|
||||
|
||||
package exceptionsapi
|
||||
|
||||
import (
|
||||
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
|
||||
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
|
||||
reflect "reflect"
|
||||
sync "sync"
|
||||
)
|
||||
|
||||
const (
|
||||
// Verify that this generated code is sufficiently up-to-date.
|
||||
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
|
||||
// Verify that runtime/protoimpl is sufficiently up-to-date.
|
||||
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
|
||||
)
|
||||
|
||||
type Exception struct {
|
||||
state protoimpl.MessageState
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
|
||||
Ecosystem string `protobuf:"bytes,2,opt,name=ecosystem,proto3" json:"ecosystem,omitempty"`
|
||||
Name string `protobuf:"bytes,3,opt,name=name,proto3" json:"name,omitempty"`
|
||||
Version string `protobuf:"bytes,4,opt,name=version,proto3" json:"version,omitempty"`
|
||||
Expires string `protobuf:"bytes,5,opt,name=expires,proto3" json:"expires,omitempty"`
|
||||
Pattern string `protobuf:"bytes,6,opt,name=pattern,proto3" json:"pattern,omitempty"` // To be used for special cases
|
||||
}
|
||||
|
||||
func (x *Exception) Reset() {
|
||||
*x = Exception{}
|
||||
if protoimpl.UnsafeEnabled {
|
||||
mi := &file_exceptions_spec_proto_msgTypes[0]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
}
|
||||
|
||||
func (x *Exception) String() string {
|
||||
return protoimpl.X.MessageStringOf(x)
|
||||
}
|
||||
|
||||
func (*Exception) ProtoMessage() {}
|
||||
|
||||
func (x *Exception) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_exceptions_spec_proto_msgTypes[0]
|
||||
if protoimpl.UnsafeEnabled && x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
return ms
|
||||
}
|
||||
return mi.MessageOf(x)
|
||||
}
|
||||
|
||||
// Deprecated: Use Exception.ProtoReflect.Descriptor instead.
|
||||
func (*Exception) Descriptor() ([]byte, []int) {
|
||||
return file_exceptions_spec_proto_rawDescGZIP(), []int{0}
|
||||
}
|
||||
|
||||
func (x *Exception) GetId() string {
|
||||
if x != nil {
|
||||
return x.Id
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Exception) GetEcosystem() string {
|
||||
if x != nil {
|
||||
return x.Ecosystem
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Exception) GetName() string {
|
||||
if x != nil {
|
||||
return x.Name
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Exception) GetVersion() string {
|
||||
if x != nil {
|
||||
return x.Version
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Exception) GetExpires() string {
|
||||
if x != nil {
|
||||
return x.Expires
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Exception) GetPattern() string {
|
||||
if x != nil {
|
||||
return x.Pattern
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
type ExceptionSuite struct {
|
||||
state protoimpl.MessageState
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
|
||||
Description string `protobuf:"bytes,2,opt,name=description,proto3" json:"description,omitempty"`
|
||||
Exceptions []*Exception `protobuf:"bytes,3,rep,name=exceptions,proto3" json:"exceptions,omitempty"`
|
||||
}
|
||||
|
||||
func (x *ExceptionSuite) Reset() {
|
||||
*x = ExceptionSuite{}
|
||||
if protoimpl.UnsafeEnabled {
|
||||
mi := &file_exceptions_spec_proto_msgTypes[1]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
}
|
||||
|
||||
func (x *ExceptionSuite) String() string {
|
||||
return protoimpl.X.MessageStringOf(x)
|
||||
}
|
||||
|
||||
func (*ExceptionSuite) ProtoMessage() {}
|
||||
|
||||
func (x *ExceptionSuite) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_exceptions_spec_proto_msgTypes[1]
|
||||
if protoimpl.UnsafeEnabled && x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
return ms
|
||||
}
|
||||
return mi.MessageOf(x)
|
||||
}
|
||||
|
||||
// Deprecated: Use ExceptionSuite.ProtoReflect.Descriptor instead.
|
||||
func (*ExceptionSuite) Descriptor() ([]byte, []int) {
|
||||
return file_exceptions_spec_proto_rawDescGZIP(), []int{1}
|
||||
}
|
||||
|
||||
func (x *ExceptionSuite) GetName() string {
|
||||
if x != nil {
|
||||
return x.Name
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *ExceptionSuite) GetDescription() string {
|
||||
if x != nil {
|
||||
return x.Description
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *ExceptionSuite) GetExceptions() []*Exception {
|
||||
if x != nil {
|
||||
return x.Exceptions
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
var File_exceptions_spec_proto protoreflect.FileDescriptor
|
||||
|
||||
var file_exceptions_spec_proto_rawDesc = []byte{
|
||||
0x0a, 0x15, 0x65, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x5f, 0x73, 0x70, 0x65,
|
||||
0x63, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x9b, 0x01, 0x0a, 0x09, 0x45, 0x78, 0x63, 0x65,
|
||||
0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28,
|
||||
0x09, 0x52, 0x02, 0x69, 0x64, 0x12, 0x1c, 0x0a, 0x09, 0x65, 0x63, 0x6f, 0x73, 0x79, 0x73, 0x74,
|
||||
0x65, 0x6d, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x65, 0x63, 0x6f, 0x73, 0x79, 0x73,
|
||||
0x74, 0x65, 0x6d, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28,
|
||||
0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69,
|
||||
0x6f, 0x6e, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f,
|
||||
0x6e, 0x12, 0x18, 0x0a, 0x07, 0x65, 0x78, 0x70, 0x69, 0x72, 0x65, 0x73, 0x18, 0x05, 0x20, 0x01,
|
||||
0x28, 0x09, 0x52, 0x07, 0x65, 0x78, 0x70, 0x69, 0x72, 0x65, 0x73, 0x12, 0x18, 0x0a, 0x07, 0x70,
|
||||
0x61, 0x74, 0x74, 0x65, 0x72, 0x6e, 0x18, 0x06, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x70, 0x61,
|
||||
0x74, 0x74, 0x65, 0x72, 0x6e, 0x22, 0x72, 0x0a, 0x0e, 0x45, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69,
|
||||
0x6f, 0x6e, 0x53, 0x75, 0x69, 0x74, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18,
|
||||
0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x20, 0x0a, 0x0b, 0x64,
|
||||
0x65, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09,
|
||||
0x52, 0x0b, 0x64, 0x65, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x2a, 0x0a,
|
||||
0x0a, 0x65, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28,
|
||||
0x0b, 0x32, 0x0a, 0x2e, 0x45, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x0a, 0x65,
|
||||
0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x42, 0x2a, 0x5a, 0x28, 0x67, 0x69, 0x74,
|
||||
0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x73, 0x61, 0x66, 0x65, 0x64, 0x65, 0x70, 0x2f,
|
||||
0x76, 0x65, 0x74, 0x2f, 0x67, 0x65, 0x6e, 0x2f, 0x65, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f,
|
||||
0x6e, 0x73, 0x61, 0x70, 0x69, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
|
||||
}
|
||||
|
||||
var (
|
||||
file_exceptions_spec_proto_rawDescOnce sync.Once
|
||||
file_exceptions_spec_proto_rawDescData = file_exceptions_spec_proto_rawDesc
|
||||
)
|
||||
|
||||
func file_exceptions_spec_proto_rawDescGZIP() []byte {
|
||||
file_exceptions_spec_proto_rawDescOnce.Do(func() {
|
||||
file_exceptions_spec_proto_rawDescData = protoimpl.X.CompressGZIP(file_exceptions_spec_proto_rawDescData)
|
||||
})
|
||||
return file_exceptions_spec_proto_rawDescData
|
||||
}
|
||||
|
||||
var file_exceptions_spec_proto_msgTypes = make([]protoimpl.MessageInfo, 2)
|
||||
var file_exceptions_spec_proto_goTypes = []interface{}{
|
||||
(*Exception)(nil), // 0: Exception
|
||||
(*ExceptionSuite)(nil), // 1: ExceptionSuite
|
||||
}
|
||||
var file_exceptions_spec_proto_depIdxs = []int32{
|
||||
0, // 0: ExceptionSuite.exceptions:type_name -> Exception
|
||||
1, // [1:1] is the sub-list for method output_type
|
||||
1, // [1:1] is the sub-list for method input_type
|
||||
1, // [1:1] is the sub-list for extension type_name
|
||||
1, // [1:1] is the sub-list for extension extendee
|
||||
0, // [0:1] is the sub-list for field type_name
|
||||
}
|
||||
|
||||
func init() { file_exceptions_spec_proto_init() }
|
||||
func file_exceptions_spec_proto_init() {
|
||||
if File_exceptions_spec_proto != nil {
|
||||
return
|
||||
}
|
||||
if !protoimpl.UnsafeEnabled {
|
||||
file_exceptions_spec_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {
|
||||
switch v := v.(*Exception); i {
|
||||
case 0:
|
||||
return &v.state
|
||||
case 1:
|
||||
return &v.sizeCache
|
||||
case 2:
|
||||
return &v.unknownFields
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
file_exceptions_spec_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {
|
||||
switch v := v.(*ExceptionSuite); i {
|
||||
case 0:
|
||||
return &v.state
|
||||
case 1:
|
||||
return &v.sizeCache
|
||||
case 2:
|
||||
return &v.unknownFields
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
type x struct{}
|
||||
out := protoimpl.TypeBuilder{
|
||||
File: protoimpl.DescBuilder{
|
||||
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
|
||||
RawDescriptor: file_exceptions_spec_proto_rawDesc,
|
||||
NumEnums: 0,
|
||||
NumMessages: 2,
|
||||
NumExtensions: 0,
|
||||
NumServices: 0,
|
||||
},
|
||||
GoTypes: file_exceptions_spec_proto_goTypes,
|
||||
DependencyIndexes: file_exceptions_spec_proto_depIdxs,
|
||||
MessageInfos: file_exceptions_spec_proto_msgTypes,
|
||||
}.Build()
|
||||
File_exceptions_spec_proto = out.File
|
||||
file_exceptions_spec_proto_rawDesc = nil
|
||||
file_exceptions_spec_proto_goTypes = nil
|
||||
file_exceptions_spec_proto_depIdxs = nil
|
||||
}
|
||||
2
go.mod
2
go.mod
@ -8,7 +8,7 @@ require (
|
||||
github.com/google/cel-go v0.13.0
|
||||
github.com/google/osv-scanner v1.1.0
|
||||
github.com/jedib0t/go-pretty/v6 v6.4.4
|
||||
github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55
|
||||
github.com/safedep/dry v0.0.0-20230222132026-c8b6cb976849
|
||||
github.com/sirupsen/logrus v1.9.0
|
||||
github.com/spf13/cobra v1.6.1
|
||||
github.com/stretchr/testify v1.8.1
|
||||
|
||||
2
go.sum
2
go.sum
@ -48,6 +48,8 @@ github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJ
|
||||
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
|
||||
github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55 h1:OBzggSWzjyEa7YaXp2DvpKDe1wYXtOEcFXQfDqkK7PI=
|
||||
github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55/go.mod h1:odFOtG1l46k23IaCY6kdNkkLW8L+NT+EUVYYVphP59I=
|
||||
github.com/safedep/dry v0.0.0-20230222132026-c8b6cb976849 h1:5nO9ht1jn7XHFyNFRhUneDZbKmwh4kRr0w/EoWuOQQA=
|
||||
github.com/safedep/dry v0.0.0-20230222132026-c8b6cb976849/go.mod h1:odFOtG1l46k23IaCY6kdNkkLW8L+NT+EUVYYVphP59I=
|
||||
github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0=
|
||||
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
|
||||
github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA=
|
||||
|
||||
24
main.go
24
main.go
@ -9,13 +9,15 @@ import (
|
||||
"github.com/safedep/dry/utils"
|
||||
"github.com/safedep/vet/internal/ui"
|
||||
"github.com/safedep/vet/pkg/common/logger"
|
||||
"github.com/safedep/vet/pkg/exceptions"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
var (
|
||||
verbose bool
|
||||
debug bool
|
||||
logFile string
|
||||
verbose bool
|
||||
debug bool
|
||||
logFile string
|
||||
globalExceptionsFile string
|
||||
)
|
||||
|
||||
var banner string = `
|
||||
@ -50,6 +52,7 @@ func main() {
|
||||
cmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "Show verbose logs")
|
||||
cmd.PersistentFlags().BoolVarP(&debug, "debug", "d", false, "Show debug logs")
|
||||
cmd.PersistentFlags().StringVarP(&logFile, "log", "l", "", "Write command logs to file")
|
||||
cmd.PersistentFlags().StringVarP(&globalExceptionsFile, "exceptions", "e", "", "Load exceptions from file")
|
||||
|
||||
cmd.AddCommand(newAuthCommand())
|
||||
cmd.AddCommand(newScanCommand())
|
||||
@ -58,6 +61,7 @@ func main() {
|
||||
|
||||
cobra.OnInitialize(func() {
|
||||
printBanner()
|
||||
loadExceptions()
|
||||
logger.SetLogLevel(verbose, debug)
|
||||
})
|
||||
|
||||
@ -66,6 +70,20 @@ func main() {
|
||||
}
|
||||
}
|
||||
|
||||
func loadExceptions() {
|
||||
if globalExceptionsFile != "" {
|
||||
loader, err := exceptions.NewExceptionsFileLoader(globalExceptionsFile)
|
||||
if err != nil {
|
||||
logger.Fatalf("Exceptions loader: %v", err)
|
||||
}
|
||||
|
||||
err = exceptions.Load(loader)
|
||||
if err != nil {
|
||||
logger.Fatalf("Exceptions loader: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func printBanner() {
|
||||
bRet, err := strconv.ParseBool(os.Getenv("VET_DISABLE_BANNER"))
|
||||
if (err != nil) || (!bRet) {
|
||||
|
||||
@ -9,6 +9,7 @@ import (
|
||||
"github.com/safedep/vet/pkg/analyzer/filter"
|
||||
"github.com/safedep/vet/pkg/common/logger"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
"github.com/safedep/vet/pkg/readers"
|
||||
)
|
||||
|
||||
type celFilterAnalyzer struct {
|
||||
@ -48,7 +49,7 @@ func (f *celFilterAnalyzer) Analyze(manifest *models.PackageManifest,
|
||||
logger.Infof("CEL filtering manifest: %s", manifest.Path)
|
||||
f.stat.IncScannedManifest()
|
||||
|
||||
for _, pkg := range manifest.Packages {
|
||||
readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
|
||||
f.stat.IncEvaluatedPackage()
|
||||
|
||||
res, err := f.evaluator.EvalPackage(pkg)
|
||||
@ -59,19 +60,21 @@ func (f *celFilterAnalyzer) Analyze(manifest *models.PackageManifest,
|
||||
pkg.PackageDetails.Name,
|
||||
pkg.PackageDetails.Version, err)
|
||||
|
||||
continue
|
||||
return nil
|
||||
}
|
||||
|
||||
if res.Matched() {
|
||||
// Avoid duplicates added to the table
|
||||
if _, ok := f.packages[pkg.Id()]; ok {
|
||||
continue
|
||||
return nil
|
||||
}
|
||||
|
||||
f.stat.IncMatchedPackage()
|
||||
f.packages[pkg.Id()] = pkg
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
|
||||
return f.notifyCaller(manifest, handler)
|
||||
}
|
||||
|
||||
@ -10,6 +10,7 @@ import (
|
||||
"github.com/safedep/vet/pkg/analyzer/filter"
|
||||
"github.com/safedep/vet/pkg/common/logger"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
"github.com/safedep/vet/pkg/readers"
|
||||
)
|
||||
|
||||
type celFilterMatchedPackage struct {
|
||||
@ -62,7 +63,7 @@ func (f *celFilterSuiteAnalyzer) Analyze(manifest *models.PackageManifest,
|
||||
logger.Infof("CEL Filter Suite: Analyzing manifest: %s", manifest.Path)
|
||||
|
||||
f.stat.IncScannedManifest()
|
||||
for _, pkg := range manifest.Packages {
|
||||
readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
|
||||
f.stat.IncEvaluatedPackage()
|
||||
|
||||
res, err := f.evaluator.EvalPackage(pkg)
|
||||
@ -73,13 +74,15 @@ func (f *celFilterSuiteAnalyzer) Analyze(manifest *models.PackageManifest,
|
||||
pkg.PackageDetails.Name,
|
||||
pkg.PackageDetails.Version, err)
|
||||
|
||||
continue
|
||||
return nil
|
||||
}
|
||||
|
||||
if res.Matched() {
|
||||
f.queueMatchedPkg(pkg, res.GetMatchedFilter().Name())
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
|
||||
if f.failOnMatch && (len(f.matchedPackages) > 0) {
|
||||
handler(&AnalyzerEvent{
|
||||
|
||||
115
pkg/analyzer/exceptions_generator.go
Normal file
115
pkg/analyzer/exceptions_generator.go
Normal file
@ -0,0 +1,115 @@
|
||||
package analyzer
|
||||
|
||||
import (
|
||||
"io"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/utils"
|
||||
"github.com/safedep/vet/gen/exceptionsapi"
|
||||
"github.com/safedep/vet/pkg/analyzer/filter"
|
||||
"github.com/safedep/vet/pkg/common/logger"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
"github.com/safedep/vet/pkg/readers"
|
||||
)
|
||||
|
||||
type exceptionsGenerator struct {
|
||||
writer io.WriteCloser
|
||||
filterEvaluator filter.Evaluator
|
||||
expires time.Time
|
||||
pkgCache map[string]*models.Package
|
||||
}
|
||||
|
||||
type ExceptionsGeneratorConfig struct {
|
||||
Path string
|
||||
Filter string
|
||||
ExpiresOn string
|
||||
}
|
||||
|
||||
func NewExceptionsGenerator(config ExceptionsGeneratorConfig) (Analyzer, error) {
|
||||
fd, err := os.Create(config.Path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
expiresOn, err := time.Parse("2006-01-02", config.ExpiresOn)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
filterEvaluator, err := filter.NewEvaluator("exceptions-generator", true)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if utils.IsEmptyString(config.Filter) {
|
||||
config.Filter = "true"
|
||||
}
|
||||
|
||||
err = filterEvaluator.AddFilter("exceptions-filter", config.Filter)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
logger.Infof("Initialized exceptions generator with filter: '%s' expiry: %s",
|
||||
config.Filter, expiresOn.Format(time.RFC3339))
|
||||
|
||||
return &exceptionsGenerator{
|
||||
writer: fd,
|
||||
filterEvaluator: filterEvaluator,
|
||||
expires: expiresOn,
|
||||
pkgCache: make(map[string]*models.Package),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (f *exceptionsGenerator) Name() string {
|
||||
return "Exceptions Generator"
|
||||
}
|
||||
|
||||
func (f *exceptionsGenerator) Analyze(manifest *models.PackageManifest,
|
||||
handler AnalyzerEventHandler) error {
|
||||
readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
|
||||
res, err := f.filterEvaluator.EvalPackage(pkg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if !res.Matched() {
|
||||
return nil
|
||||
}
|
||||
|
||||
f.pkgCache[pkg.Id()] = pkg
|
||||
return nil
|
||||
})
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (f *exceptionsGenerator) Finish() error {
|
||||
defer f.writer.Close()
|
||||
|
||||
suite := exceptionsapi.ExceptionSuite{
|
||||
Name: "Auto Generated Exceptions",
|
||||
Description: "Exceptions file auto-generated using vet",
|
||||
Exceptions: make([]*exceptionsapi.Exception, 0),
|
||||
}
|
||||
|
||||
for _, pkg := range f.pkgCache {
|
||||
logger.Infof("Adding %s to exceptions list", pkg.ShortName())
|
||||
|
||||
suite.Exceptions = append(suite.Exceptions, &exceptionsapi.Exception{
|
||||
Id: utils.NewUniqueId(),
|
||||
Ecosystem: string(pkg.Ecosystem),
|
||||
Name: pkg.Name,
|
||||
Version: pkg.Version,
|
||||
Expires: f.expires.Format(time.RFC3339),
|
||||
})
|
||||
}
|
||||
|
||||
err := utils.FromPbToYaml(f.writer, &suite)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@ -51,6 +51,10 @@ func Debugf(msg string, args ...any) {
|
||||
logrus.Debugf(msg, args...)
|
||||
}
|
||||
|
||||
func Fatalf(msg string, args ...any) {
|
||||
logrus.Fatalf(msg, args...)
|
||||
}
|
||||
|
||||
func LoggerWith(key string, value any) *logrus.Entry {
|
||||
return logrus.WithFields(logrus.Fields{
|
||||
key: value,
|
||||
|
||||
148
pkg/exceptions/exceptions.go
Normal file
148
pkg/exceptions/exceptions.go
Normal file
@ -0,0 +1,148 @@
|
||||
package exceptions
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"hash/fnv"
|
||||
"io"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/vet/gen/exceptionsapi"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
)
|
||||
|
||||
var (
|
||||
jitter = 5 * time.Second
|
||||
)
|
||||
|
||||
// Represents an exception rule as per spec with additional details
|
||||
type exceptionRule struct {
|
||||
spec *exceptionsapi.Exception
|
||||
expiry time.Time
|
||||
}
|
||||
|
||||
// In-memory store of exceptions to be used for package hash and exception ID
|
||||
// based lookup for fast matching and avoid duplicates
|
||||
type exceptionStore struct {
|
||||
m sync.RWMutex
|
||||
rules map[string]map[string]*exceptionRule
|
||||
}
|
||||
|
||||
// Represents an exceptions loader interface to support loading exceptions
|
||||
// from multiple sources
|
||||
type exceptionsLoader interface {
|
||||
// Read an exception rule, return io.EOF on done
|
||||
Read() (*exceptionRule, error)
|
||||
}
|
||||
|
||||
// Represents an exception match result
|
||||
type exceptionMatchResult struct {
|
||||
pkg *models.Package
|
||||
rule *exceptionRule
|
||||
}
|
||||
|
||||
// Global exceptions store
|
||||
var globalExceptions *exceptionStore
|
||||
|
||||
// Initialize the global exceptions cache
|
||||
func init() {
|
||||
initStore()
|
||||
}
|
||||
|
||||
func initStore() {
|
||||
globalExceptions = &exceptionStore{
|
||||
rules: make(map[string]map[string]*exceptionRule),
|
||||
}
|
||||
}
|
||||
|
||||
func Load(loader exceptionsLoader) error {
|
||||
globalExceptions.m.Lock()
|
||||
defer globalExceptions.m.Unlock()
|
||||
|
||||
for {
|
||||
rule, err := loader.Read()
|
||||
if err != nil {
|
||||
if err == io.EOF {
|
||||
break
|
||||
} else {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if rule.expiry.Before(time.Now().Add(jitter)) {
|
||||
continue
|
||||
}
|
||||
|
||||
h := pkgHash(rule.spec.GetEcosystem(), rule.spec.GetName())
|
||||
if _, ok := globalExceptions.rules[h]; ok {
|
||||
if _, ok = globalExceptions.rules[h][rule.spec.GetId()]; ok {
|
||||
continue
|
||||
}
|
||||
} else {
|
||||
globalExceptions.rules[h] = make(map[string]*exceptionRule)
|
||||
}
|
||||
|
||||
globalExceptions.rules[h][rule.spec.GetId()] = rule
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func Apply(pkg *models.Package) (*exceptionMatchResult, error) {
|
||||
return globalExceptions.Match(pkg)
|
||||
}
|
||||
|
||||
func ActiveCount() int {
|
||||
return globalExceptions.ActiveCount()
|
||||
}
|
||||
|
||||
func (s *exceptionStore) ActiveCount() int {
|
||||
return len(s.rules)
|
||||
}
|
||||
|
||||
func (s *exceptionStore) Match(pkg *models.Package) (*exceptionMatchResult, error) {
|
||||
result := exceptionMatchResult{}
|
||||
|
||||
s.m.RLock()
|
||||
defer s.m.RUnlock()
|
||||
|
||||
h := pkgHash(string(pkg.PackageDetails.Ecosystem), pkg.PackageDetails.Name)
|
||||
if _, ok := s.rules[h]; !ok {
|
||||
return &result, nil
|
||||
}
|
||||
|
||||
for _, rule := range s.rules[h] {
|
||||
if rule.matchByPattern(pkg) || rule.matchByVersion(pkg) {
|
||||
result.pkg = pkg
|
||||
result.rule = rule
|
||||
|
||||
return &result, nil
|
||||
}
|
||||
}
|
||||
|
||||
return &result, nil
|
||||
}
|
||||
|
||||
func (r *exceptionRule) matchByPattern(pkg *models.Package) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
func (r *exceptionRule) matchByVersion(pkg *models.Package) bool {
|
||||
return strings.EqualFold(string(pkg.PackageDetails.Ecosystem), r.spec.GetEcosystem()) &&
|
||||
strings.EqualFold(pkg.PackageDetails.Name, r.spec.GetName()) &&
|
||||
((r.spec.GetVersion() == "*") || (r.spec.GetVersion() == pkg.PackageDetails.Version))
|
||||
}
|
||||
|
||||
func (r *exceptionMatchResult) Matched() bool {
|
||||
return (r == nil) || ((r.pkg != nil) && (r.rule != nil))
|
||||
}
|
||||
|
||||
func pkgHash(ecosystem, name string) string {
|
||||
h := fnv.New64a()
|
||||
h.Write([]byte(fmt.Sprintf("%s/%s",
|
||||
strings.ToLower(ecosystem), strings.ToLower(name))))
|
||||
|
||||
return strconv.FormatUint(h.Sum64(), 16)
|
||||
}
|
||||
60
pkg/exceptions/exceptions_loader_file.go
Normal file
60
pkg/exceptions/exceptions_loader_file.go
Normal file
@ -0,0 +1,60 @@
|
||||
package exceptions
|
||||
|
||||
import (
|
||||
"io"
|
||||
"os"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/utils"
|
||||
"github.com/safedep/vet/gen/exceptionsapi"
|
||||
)
|
||||
|
||||
type exceptionsFileLoader struct {
|
||||
m sync.Mutex
|
||||
idx int
|
||||
suite *exceptionsapi.ExceptionSuite
|
||||
}
|
||||
|
||||
func NewExceptionsFileLoader(path string) (exceptionsLoader, error) {
|
||||
file, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return newExceptionsFileLoaderUsingReader(file)
|
||||
}
|
||||
|
||||
func newExceptionsFileLoaderUsingReader(reader io.Reader) (exceptionsLoader, error) {
|
||||
var suite exceptionsapi.ExceptionSuite
|
||||
err := utils.FromYamlToPb(reader, &suite)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &exceptionsFileLoader{
|
||||
suite: &suite,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (f *exceptionsFileLoader) Read() (*exceptionRule, error) {
|
||||
f.m.Lock()
|
||||
defer f.m.Unlock()
|
||||
|
||||
if f.idx >= len(f.suite.Exceptions) {
|
||||
return nil, io.EOF
|
||||
}
|
||||
|
||||
cx := f.suite.Exceptions[f.idx]
|
||||
f.idx += 1
|
||||
|
||||
expiry, err := time.Parse(time.RFC3339, cx.Expires)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &exceptionRule{
|
||||
spec: cx,
|
||||
expiry: expiry,
|
||||
}, nil
|
||||
}
|
||||
86
pkg/exceptions/exceptions_loader_file_test.go
Normal file
86
pkg/exceptions/exceptions_loader_file_test.go
Normal file
@ -0,0 +1,86 @@
|
||||
package exceptions
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestNewExceptionsFileLoader(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
file string
|
||||
ecosystem string
|
||||
pkgName string
|
||||
version string
|
||||
match bool
|
||||
errMsg string
|
||||
}{
|
||||
{
|
||||
"Load a file and match",
|
||||
"fixtures/1_valid.yml",
|
||||
"maven",
|
||||
"p1",
|
||||
"v1",
|
||||
true,
|
||||
"",
|
||||
},
|
||||
{
|
||||
"Match any version",
|
||||
"fixtures/1_valid.yml",
|
||||
"maven",
|
||||
"p2",
|
||||
"v5-anything",
|
||||
true,
|
||||
"",
|
||||
},
|
||||
{
|
||||
"Does not match expired exceptions",
|
||||
"fixtures/1_valid.yml",
|
||||
"maven",
|
||||
"p3",
|
||||
"v5-anything",
|
||||
false,
|
||||
"",
|
||||
},
|
||||
{
|
||||
"Error loading file that does not exist",
|
||||
"fixtures/1_does_not_exists.yml",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
false,
|
||||
"no such file or directory",
|
||||
},
|
||||
{
|
||||
"Error loading invalid YAML file",
|
||||
"fixtures/2_invalid.yml",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
false,
|
||||
"unknown field",
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range cases {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
initStore()
|
||||
|
||||
loader, err := NewExceptionsFileLoader(test.file)
|
||||
if test.errMsg != "" {
|
||||
assert.NotNil(t, err)
|
||||
assert.ErrorContains(t, err, test.errMsg)
|
||||
return
|
||||
}
|
||||
|
||||
Load(loader)
|
||||
|
||||
pd := models.NewPackageDetail(test.ecosystem, test.pkgName, test.version)
|
||||
res, _ := Apply(&models.Package{PackageDetails: pd})
|
||||
|
||||
assert.Equal(t, test.match, res.Matched())
|
||||
})
|
||||
}
|
||||
}
|
||||
252
pkg/exceptions/exceptions_test.go
Normal file
252
pkg/exceptions/exceptions_test.go
Normal file
@ -0,0 +1,252 @@
|
||||
package exceptions
|
||||
|
||||
import (
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/vet/gen/exceptionsapi"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
type exceptionsLoaderMocker struct {
|
||||
rules []exceptionRule
|
||||
idx int
|
||||
}
|
||||
|
||||
func (m *exceptionsLoaderMocker) Read() (*exceptionRule, error) {
|
||||
if m.idx >= len(m.rules) {
|
||||
return nil, io.EOF
|
||||
}
|
||||
|
||||
r := m.rules[m.idx]
|
||||
m.idx += 1
|
||||
|
||||
return &r, nil
|
||||
}
|
||||
|
||||
func TestLoad(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
rules []exceptionRule
|
||||
pCount int // Package count
|
||||
}{
|
||||
{
|
||||
"Load a rule",
|
||||
[]exceptionRule{
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "a",
|
||||
Ecosystem: "maven",
|
||||
Name: "p1",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
},
|
||||
1,
|
||||
},
|
||||
{
|
||||
"Load two rule for same package",
|
||||
[]exceptionRule{
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "a",
|
||||
Ecosystem: "maven",
|
||||
Name: "p1",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "b",
|
||||
Ecosystem: "maven",
|
||||
Name: "p1",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
},
|
||||
1,
|
||||
},
|
||||
{
|
||||
"Load two rule for two package",
|
||||
[]exceptionRule{
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "a",
|
||||
Ecosystem: "maven",
|
||||
Name: "p1",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "b",
|
||||
Ecosystem: "maven",
|
||||
Name: "p2",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
},
|
||||
2,
|
||||
},
|
||||
{
|
||||
"Load an expired rule",
|
||||
[]exceptionRule{
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "a",
|
||||
Ecosystem: "maven",
|
||||
Name: "p1",
|
||||
},
|
||||
expiry: time.Now().Add(-1 * time.Hour),
|
||||
},
|
||||
},
|
||||
0,
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range cases {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
// Reset the store
|
||||
initStore()
|
||||
|
||||
Load(&exceptionsLoaderMocker{
|
||||
rules: test.rules,
|
||||
})
|
||||
|
||||
assert.Equal(t, test.pCount, len(globalExceptions.rules))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestApply(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
rules []exceptionRule
|
||||
ecosystem string
|
||||
pkgName string
|
||||
version string
|
||||
match bool
|
||||
errMsg string
|
||||
}{
|
||||
{
|
||||
"Match by version",
|
||||
[]exceptionRule{
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "a",
|
||||
Ecosystem: "maven",
|
||||
Name: "p1",
|
||||
Version: "v1",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
},
|
||||
"maven",
|
||||
"p1",
|
||||
"v1",
|
||||
true,
|
||||
"",
|
||||
},
|
||||
{
|
||||
"Match any version",
|
||||
[]exceptionRule{
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "a",
|
||||
Ecosystem: "maven",
|
||||
Name: "p1",
|
||||
Version: "*",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
},
|
||||
"maven",
|
||||
"p1",
|
||||
"v-anything",
|
||||
true,
|
||||
"",
|
||||
},
|
||||
{
|
||||
"No Match without a version",
|
||||
[]exceptionRule{
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "a",
|
||||
Ecosystem: "maven",
|
||||
Name: "p1",
|
||||
Version: "",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
},
|
||||
"maven",
|
||||
"p1",
|
||||
"v-anything",
|
||||
false,
|
||||
"",
|
||||
},
|
||||
{
|
||||
"Match with case-insensitive",
|
||||
[]exceptionRule{
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "a",
|
||||
Ecosystem: "MAVEN",
|
||||
Name: "P1",
|
||||
Version: "v1",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
},
|
||||
"maven",
|
||||
"p1",
|
||||
"v1",
|
||||
true,
|
||||
"",
|
||||
},
|
||||
{
|
||||
"No match with case-insensitive version",
|
||||
[]exceptionRule{
|
||||
{
|
||||
spec: &exceptionsapi.Exception{
|
||||
Id: "a",
|
||||
Ecosystem: "MAVEN",
|
||||
Name: "P1",
|
||||
Version: "V1",
|
||||
},
|
||||
expiry: time.Now().Add(1 * time.Hour),
|
||||
},
|
||||
},
|
||||
"maven",
|
||||
"p1",
|
||||
"v1",
|
||||
false,
|
||||
"",
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range cases {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
initStore()
|
||||
|
||||
Load(&exceptionsLoaderMocker{
|
||||
rules: test.rules,
|
||||
})
|
||||
|
||||
pd := models.NewPackageDetail(test.ecosystem, test.pkgName, test.version)
|
||||
res, err := Apply(&models.Package{
|
||||
PackageDetails: pd,
|
||||
})
|
||||
|
||||
if test.errMsg != "" {
|
||||
assert.NotNil(t, err)
|
||||
assert.ErrorContains(t, err, test.errMsg)
|
||||
} else {
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, test.match, res.Matched())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
17
pkg/exceptions/fixtures/1_valid.yml
Normal file
17
pkg/exceptions/fixtures/1_valid.yml
Normal file
@ -0,0 +1,17 @@
|
||||
name: Exceptions File for Fixture
|
||||
description: |
|
||||
Test exceptions file
|
||||
exceptions:
|
||||
- ecosystem: maven
|
||||
name: p1
|
||||
version: v1
|
||||
expires: 2050-11-10T23:00:00Z
|
||||
- ecosystem: maven
|
||||
name: p2
|
||||
version: '*'
|
||||
expires: 2050-11-10T23:00:00Z
|
||||
- ecosystem: maven
|
||||
name: p3
|
||||
version: '*'
|
||||
expires: 2020-11-10T23:00:00Z
|
||||
|
||||
2
pkg/exceptions/fixtures/2_invalid.yml
Normal file
2
pkg/exceptions/fixtures/2_invalid.yml
Normal file
@ -0,0 +1,2 @@
|
||||
A: 1
|
||||
B: 2
|
||||
33
pkg/exceptions/utils.go
Normal file
33
pkg/exceptions/utils.go
Normal file
@ -0,0 +1,33 @@
|
||||
package exceptions
|
||||
|
||||
import (
|
||||
"github.com/safedep/vet/pkg/common/logger"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
)
|
||||
|
||||
// AllowedPackages iterates over packages in the manifest and call handler
|
||||
// only for packages not in the exempted by exception rules
|
||||
func AllowedPackages(manifest *models.PackageManifest,
|
||||
handler func(pkg *models.Package) error) error {
|
||||
for _, pkg := range manifest.Packages {
|
||||
res, err := Apply(pkg)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to evaluate exception for %s: %v",
|
||||
pkg.ShortName(), err)
|
||||
continue
|
||||
}
|
||||
|
||||
if res.Matched() {
|
||||
logger.Debugf("Ignoring package:%s due to exception rule",
|
||||
pkg.ShortName())
|
||||
continue
|
||||
}
|
||||
|
||||
err = handler(pkg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@ -75,6 +75,12 @@ func (p *Package) Id() string {
|
||||
return strconv.FormatUint(h.Sum64(), 16)
|
||||
}
|
||||
|
||||
func (p *Package) ShortName() string {
|
||||
return fmt.Sprintf("pkg:%s/%s@%s",
|
||||
strings.ToLower(string(p.Ecosystem)),
|
||||
strings.ToLower(p.Name), p.Version)
|
||||
}
|
||||
|
||||
func NewPackageDetail(e, n, v string) lockfile.PackageDetails {
|
||||
return lockfile.PackageDetails{
|
||||
Ecosystem: lockfile.Ecosystem(e),
|
||||
|
||||
18
pkg/readers/pm.go
Normal file
18
pkg/readers/pm.go
Normal file
@ -0,0 +1,18 @@
|
||||
package readers
|
||||
|
||||
import (
|
||||
"github.com/safedep/vet/pkg/exceptions"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
)
|
||||
|
||||
type packageManifestReader struct {
|
||||
manifest *models.PackageManifest
|
||||
}
|
||||
|
||||
func NewManifestModelReader(manifest *models.PackageManifest) PackageReader {
|
||||
return &packageManifestReader{manifest: manifest}
|
||||
}
|
||||
|
||||
func (r *packageManifestReader) EnumPackages(handler func(pkg *models.Package) error) error {
|
||||
return exceptions.AllowedPackages(r.manifest, handler)
|
||||
}
|
||||
16
pkg/readers/reader.go
Normal file
16
pkg/readers/reader.go
Normal file
@ -0,0 +1,16 @@
|
||||
package readers
|
||||
|
||||
import "github.com/safedep/vet/pkg/models"
|
||||
|
||||
// Contract for implementing package manifest readers such as lockfile parser,
|
||||
// SBOM parser etc. Reader should stop enumeration and return error if handler
|
||||
// returns an error
|
||||
type PackageManifestReader interface {
|
||||
EnumManifests(func(*models.PackageManifest) error) error
|
||||
}
|
||||
|
||||
// Contract for implementing a package reader. Enumerator should fail and return
|
||||
// error if handler fails
|
||||
type PackageReader interface {
|
||||
EnumPackages(func(*models.Package) error) error
|
||||
}
|
||||
@ -11,6 +11,7 @@ import (
|
||||
"github.com/safedep/vet/pkg/analyzer"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
"github.com/safedep/vet/pkg/policy"
|
||||
"github.com/safedep/vet/pkg/readers"
|
||||
)
|
||||
|
||||
type consoleReporter struct{}
|
||||
@ -29,9 +30,10 @@ func (r *consoleReporter) AddManifest(manifest *models.PackageManifest) {
|
||||
tbl.SetStyle(table.StyleLight)
|
||||
|
||||
tbl.AppendHeader(table.Row{"Package", "Attribute", "Summary"})
|
||||
for _, pkg := range manifest.Packages {
|
||||
readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
|
||||
r.report(tbl, pkg)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
|
||||
fmt.Print(text.Bold.Sprint("Manifest: ", text.FgBlue.Sprint(manifest.Path)))
|
||||
fmt.Print("\n")
|
||||
|
||||
@ -12,8 +12,10 @@ import (
|
||||
"github.com/safedep/dry/utils"
|
||||
"github.com/safedep/vet/gen/insightapi"
|
||||
"github.com/safedep/vet/pkg/analyzer"
|
||||
"github.com/safedep/vet/pkg/exceptions"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
"github.com/safedep/vet/pkg/policy"
|
||||
"github.com/safedep/vet/pkg/readers"
|
||||
)
|
||||
|
||||
const (
|
||||
@ -72,13 +74,14 @@ func (r *summaryReporter) Name() string {
|
||||
}
|
||||
|
||||
func (r *summaryReporter) AddManifest(manifest *models.PackageManifest) {
|
||||
for _, pkg := range manifest.Packages {
|
||||
readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
|
||||
r.processForVulns(pkg)
|
||||
r.processForPopularity(pkg)
|
||||
r.processForVersionDrift(pkg)
|
||||
|
||||
r.summary.packages += 1
|
||||
}
|
||||
return nil
|
||||
})
|
||||
|
||||
r.summary.manifests += 1
|
||||
}
|
||||
@ -196,9 +199,14 @@ func (r *summaryReporter) Finish() error {
|
||||
r.renderRemediationAdvice()
|
||||
fmt.Println()
|
||||
|
||||
if exceptions.ActiveCount() > 0 {
|
||||
fmt.Println(text.Faint.Sprint(summaryListPrependText, r.exceptionsCountStatement()))
|
||||
fmt.Println()
|
||||
}
|
||||
|
||||
fmt.Println("Run with `vet --filter=\"...\"` for custom filters to identify risky libraries")
|
||||
fmt.Println()
|
||||
fmt.Println("For more details", text.Bold.Sprint("https://github.com/safedep/vet"))
|
||||
fmt.Println()
|
||||
|
||||
return nil
|
||||
}
|
||||
@ -296,3 +304,8 @@ func (r *summaryReporter) majorVersionDriftStatement() string {
|
||||
return fmt.Sprintf("%d libraries are out of date with major version drift in direct dependencies",
|
||||
r.summary.metrics.drifts)
|
||||
}
|
||||
|
||||
func (r *summaryReporter) exceptionsCountStatement() string {
|
||||
return fmt.Sprintf("%d libraries are exempted from analysis through exception rules",
|
||||
exceptions.ActiveCount())
|
||||
}
|
||||
|
||||
@ -7,6 +7,7 @@ import (
|
||||
"github.com/safedep/vet/pkg/common/logger"
|
||||
"github.com/safedep/vet/pkg/common/utils"
|
||||
"github.com/safedep/vet/pkg/models"
|
||||
"github.com/safedep/vet/pkg/readers"
|
||||
"github.com/safedep/vet/pkg/reporter"
|
||||
)
|
||||
|
||||
@ -217,9 +218,10 @@ func (s *packageManifestScanner) enrichManifest(manifest *models.PackageManifest
|
||||
|
||||
q.Start()
|
||||
|
||||
for _, pkg := range manifest.Packages {
|
||||
readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
|
||||
q.Add(pkg)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
|
||||
q.Wait()
|
||||
q.Stop()
|
||||
|
||||
28
query.go
28
query.go
@ -1,6 +1,8 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/utils"
|
||||
"github.com/safedep/vet/pkg/analyzer"
|
||||
"github.com/safedep/vet/pkg/reporter"
|
||||
@ -16,6 +18,11 @@ var (
|
||||
queryEnableConsoleReport bool
|
||||
queryEnableSummaryReport bool
|
||||
queryMarkdownReportPath string
|
||||
queryExceptionsFile string
|
||||
queryExceptionsTill string
|
||||
queryExceptionsFilter string
|
||||
|
||||
queryDefaultExceptionExpiry = time.Now().Add(90 * 24 * time.Hour)
|
||||
)
|
||||
|
||||
func newQueryCommand() *cobra.Command {
|
||||
@ -36,6 +43,13 @@ func newQueryCommand() *cobra.Command {
|
||||
"Filter packages using CEL Filter Suite from file")
|
||||
cmd.Flags().BoolVarP(&queryFilterFailOnMatch, "filter-fail", "", false,
|
||||
"Fail the command if filter matches any package (for security gate)")
|
||||
cmd.Flags().StringVarP(&queryExceptionsFile, "exceptions-generate", "", "",
|
||||
"Generate exception records to file (YAML)")
|
||||
cmd.Flags().StringVarP(&queryExceptionsTill, "exceptions-till", "",
|
||||
queryDefaultExceptionExpiry.Format("2006-01-02"),
|
||||
"Generated exceptions are valid till")
|
||||
cmd.Flags().StringVarP(&queryExceptionsFilter, "exceptions-filter", "", "",
|
||||
"Generate exception records for packages matching filter")
|
||||
cmd.Flags().BoolVarP(&queryEnableConsoleReport, "report-console", "", false,
|
||||
"Minimal summary of package manifest")
|
||||
cmd.Flags().BoolVarP(&queryEnableSummaryReport, "report-summary", "", false,
|
||||
@ -74,6 +88,20 @@ func internalStartQuery() error {
|
||||
analyzers = append(analyzers, task)
|
||||
}
|
||||
|
||||
if !utils.IsEmptyString(queryExceptionsFile) {
|
||||
task, err := analyzer.NewExceptionsGenerator(analyzer.ExceptionsGeneratorConfig{
|
||||
Path: queryExceptionsFile,
|
||||
ExpiresOn: queryExceptionsTill,
|
||||
Filter: queryExceptionsFilter,
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
analyzers = append(analyzers, task)
|
||||
}
|
||||
|
||||
if queryEnableConsoleReport {
|
||||
rp, err := reporter.NewConsoleReporter()
|
||||
if err != nil {
|
||||
|
||||
13
samples/exceptions.yml
Normal file
13
samples/exceptions.yml
Normal file
@ -0,0 +1,13 @@
|
||||
name: Sample Exceptions File
|
||||
description: |
|
||||
This is a sample exceptions file
|
||||
exceptions:
|
||||
- ecosystem: maven
|
||||
name: com.example.mypackage:mylib1
|
||||
version: '1.2.3'
|
||||
expires: 2009-11-10T23:00:00Z
|
||||
- ecosystem: maven
|
||||
name: com.example.mypackage:mylib2
|
||||
version: '*'
|
||||
expires: 2009-11-10T23:00:00Z
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user