Merge pull request #25 from safedep/develop

Add Support for Package Exceptions
This commit is contained in:
Abhisek Datta 2023-02-23 10:38:42 +05:30 committed by GitHub
commit 56fdeb64ce
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
28 changed files with 1278 additions and 19 deletions

View File

@ -25,9 +25,18 @@ protoc-codegen:
--go_out=./gen/filtersuite \ --go_out=./gen/filtersuite \
--go_opt=paths=source_relative \ --go_opt=paths=source_relative \
./api/filter_suite_spec.proto ./api/filter_suite_spec.proto
protoc -I ./api \
--go_out=./gen/exceptionsapi \
--go_opt=paths=source_relative \
./api/exceptions_spec.proto
setup: setup:
mkdir -p out gen/insightapi gen/controlplane gen/filterinput gen/filtersuite mkdir -p out \
gen/insightapi \
gen/controlplane \
gen/filterinput \
gen/filtersuite \
gen/exceptionsapi
GO_CFLAGS=-X main.commit=$(GITCOMMIT) -X main.version=$(VERSION) GO_CFLAGS=-X main.commit=$(GITCOMMIT) -X main.version=$(VERSION)
GO_LDFLAGS=-ldflags "-w $(GO_CFLAGS)" GO_LDFLAGS=-ldflags "-w $(GO_CFLAGS)"

View File

@ -14,6 +14,17 @@ source dependencies and evaluate them against organizational policies.
## TL;DR ## TL;DR
Scan a repository for OSS dependency risks with auto-detection of package
manifests
```bash
vet scan -D /path/to/repo
```
![vet Summary Demo](docs/images/vet-summary-demo.png)
## Getting Started
> Ensure `$(go env GOPATH)/bin` is in your `$PATH` > Ensure `$(go env GOPATH)/bin` is in your `$PATH`
Install using `go get` Install using `go get`
@ -57,6 +68,12 @@ vet scan --lockfiles /path/to/requirements.txt
vet scan --lockfiles /path/to/package-lock.json vet scan --lockfiles /path/to/package-lock.json
``` ```
or scan a supported package manifest with a non-standard name
```bash
vet scan --lockfiles /path/to/gradle-compileOnly.lock --lockfile-as gradle.lockfile
```
> Use `vet scan parsers` to list supported package manifest parsers > Use `vet scan parsers` to list supported package manifest parsers
The default scan uses an opinionated [Summary Reporter](#) which presents The default scan uses an opinionated [Summary Reporter](#) which presents
@ -98,6 +115,21 @@ vet scan -D /path/to/dir --filter-suite /path/to/suite.yml --filter-fail
Read more about filter suites in [filtering guide](docs/filtering.md) Read more about filter suites in [filtering guide](docs/filtering.md)
## Exceptions Management
Exception rules can be generated using the `query` workflow to temporarily
ignore (or snooze) existing issues when using `vet` for the first time. This
helps in establishing security gating to prevent introduction of new security
issues while existing issues are being remediated.
Use exception rules during scan to ignore specific packages
```bash
vet scan -D /path/to/repo -e /path/to/exceptions.yml
```
For more information, refer to [exceptions guide](docs/exceptions.md)
## FAQ ## FAQ
### How do I disable the stupid banner? ### How do I disable the stupid banner?

18
api/exceptions_spec.proto Normal file
View File

@ -0,0 +1,18 @@
syntax = "proto3";
option go_package = "github.com/safedep/vet/gen/exceptionsapi";
message Exception {
string id = 1;
string ecosystem = 2;
string name = 3;
string version = 4;
string expires = 5;
string pattern = 6; // To be used for special cases
}
message ExceptionSuite {
string name = 1;
string description = 2;
repeated Exception exceptions = 3;
}

84
docs/exceptions.md Normal file
View File

@ -0,0 +1,84 @@
# Exceptions Management
Any security scanning tool may produce
1. False positive
2. Issues that are acceptable for a period of time
3. Issues that are ignored permanently
To support exceptions, we introduce the exception model defined in [exception
spec](../api/exceptions_spec.proto)
## Use-case
> As a user of `vet` tool, I want to add all existing packages or package
> versions as `exceptions` to make the scanner and filter analyser to ignore
> them while reporting issues so that I can deploy `vet` as a security gate to
> prevent introducing new packages with security issues
This workflow will allow users to
1. Accept the current issues as backlog to be mitigated over time
2. Deploy `vet` as a security gate in CI to prevent introducing new issues
### Security Risks
Exceptions management should handle the potential security risk of ignoring a
package and its future issues. To mitigate this risk, we will ensure that issues
can be ignored till an acceptable time window and not permanently.
## Workflow
### Generate Exceptions File
Run a scan and dump raw data to a directory
```bash
vet scan -D /path/to/repo --json-dump-dir /path/to/dump
```
Use `vet query` to generate exceptions for all existing packages
```bash
vet query --from /path/to/dump \
--exceptions-generate /path/to/exceptions.yml \
--exceptions-filter 'true' \ # Optional filter for packages to add
--exceptions-till '2023-12-12'
```
> `--exceptions-till` is parsed as `YYYY-mm-dd` and will generate a timestamp
> of `00:00:00` in UTC timezone for the date in RFC3339 format
### Customize Exceptions File
The generated exceptions file will add all packages, matching optional filter,
into the `exceptions.yml` file. This file should be reviewed and customised as
required before using it.
### Use Exceptions to Ignore Specific Packages
An exceptions file can be passed as a global flag to `vet`. It will be used for
various commands such as `scan` or `query`.
```bash
./vet --exceptions /path/to/exceptions.yml scan -D /path/to/repo
```
> **Note:** Do not pass this flag while generating exceptions list in query
> workflow to avoid incorrect exception list generation
## Behaviour
* All exceptions rules are applied only on a `Package`
* All comparisons will be case-insensitive except version
* Only `version` can have a value of `*` matching any version
* Exceptions are globally managed and will be shared across packages
* Exempted packages will be ignored by all analysers and reporters
* First match policy for exceptions matching
Anti-patterns that will NOT be implemented
* Exceptions will not be implemented for manifests because they will cause
false negatives
* Exceptions will not be created without an expiry to avoid future false
negatives on the package

Binary file not shown.

After

Width:  |  Height:  |  Size: 82 KiB

View File

@ -0,0 +1,275 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.28.0
// protoc v3.18.0
// source: exceptions_spec.proto
package exceptionsapi
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type Exception struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
Ecosystem string `protobuf:"bytes,2,opt,name=ecosystem,proto3" json:"ecosystem,omitempty"`
Name string `protobuf:"bytes,3,opt,name=name,proto3" json:"name,omitempty"`
Version string `protobuf:"bytes,4,opt,name=version,proto3" json:"version,omitempty"`
Expires string `protobuf:"bytes,5,opt,name=expires,proto3" json:"expires,omitempty"`
Pattern string `protobuf:"bytes,6,opt,name=pattern,proto3" json:"pattern,omitempty"` // To be used for special cases
}
func (x *Exception) Reset() {
*x = Exception{}
if protoimpl.UnsafeEnabled {
mi := &file_exceptions_spec_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *Exception) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Exception) ProtoMessage() {}
func (x *Exception) ProtoReflect() protoreflect.Message {
mi := &file_exceptions_spec_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Exception.ProtoReflect.Descriptor instead.
func (*Exception) Descriptor() ([]byte, []int) {
return file_exceptions_spec_proto_rawDescGZIP(), []int{0}
}
func (x *Exception) GetId() string {
if x != nil {
return x.Id
}
return ""
}
func (x *Exception) GetEcosystem() string {
if x != nil {
return x.Ecosystem
}
return ""
}
func (x *Exception) GetName() string {
if x != nil {
return x.Name
}
return ""
}
func (x *Exception) GetVersion() string {
if x != nil {
return x.Version
}
return ""
}
func (x *Exception) GetExpires() string {
if x != nil {
return x.Expires
}
return ""
}
func (x *Exception) GetPattern() string {
if x != nil {
return x.Pattern
}
return ""
}
type ExceptionSuite struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
Description string `protobuf:"bytes,2,opt,name=description,proto3" json:"description,omitempty"`
Exceptions []*Exception `protobuf:"bytes,3,rep,name=exceptions,proto3" json:"exceptions,omitempty"`
}
func (x *ExceptionSuite) Reset() {
*x = ExceptionSuite{}
if protoimpl.UnsafeEnabled {
mi := &file_exceptions_spec_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *ExceptionSuite) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*ExceptionSuite) ProtoMessage() {}
func (x *ExceptionSuite) ProtoReflect() protoreflect.Message {
mi := &file_exceptions_spec_proto_msgTypes[1]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use ExceptionSuite.ProtoReflect.Descriptor instead.
func (*ExceptionSuite) Descriptor() ([]byte, []int) {
return file_exceptions_spec_proto_rawDescGZIP(), []int{1}
}
func (x *ExceptionSuite) GetName() string {
if x != nil {
return x.Name
}
return ""
}
func (x *ExceptionSuite) GetDescription() string {
if x != nil {
return x.Description
}
return ""
}
func (x *ExceptionSuite) GetExceptions() []*Exception {
if x != nil {
return x.Exceptions
}
return nil
}
var File_exceptions_spec_proto protoreflect.FileDescriptor
var file_exceptions_spec_proto_rawDesc = []byte{
0x0a, 0x15, 0x65, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x5f, 0x73, 0x70, 0x65,
0x63, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x9b, 0x01, 0x0a, 0x09, 0x45, 0x78, 0x63, 0x65,
0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28,
0x09, 0x52, 0x02, 0x69, 0x64, 0x12, 0x1c, 0x0a, 0x09, 0x65, 0x63, 0x6f, 0x73, 0x79, 0x73, 0x74,
0x65, 0x6d, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x65, 0x63, 0x6f, 0x73, 0x79, 0x73,
0x74, 0x65, 0x6d, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28,
0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69,
0x6f, 0x6e, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f,
0x6e, 0x12, 0x18, 0x0a, 0x07, 0x65, 0x78, 0x70, 0x69, 0x72, 0x65, 0x73, 0x18, 0x05, 0x20, 0x01,
0x28, 0x09, 0x52, 0x07, 0x65, 0x78, 0x70, 0x69, 0x72, 0x65, 0x73, 0x12, 0x18, 0x0a, 0x07, 0x70,
0x61, 0x74, 0x74, 0x65, 0x72, 0x6e, 0x18, 0x06, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x70, 0x61,
0x74, 0x74, 0x65, 0x72, 0x6e, 0x22, 0x72, 0x0a, 0x0e, 0x45, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69,
0x6f, 0x6e, 0x53, 0x75, 0x69, 0x74, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18,
0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x20, 0x0a, 0x0b, 0x64,
0x65, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09,
0x52, 0x0b, 0x64, 0x65, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x2a, 0x0a,
0x0a, 0x65, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28,
0x0b, 0x32, 0x0a, 0x2e, 0x45, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x0a, 0x65,
0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x42, 0x2a, 0x5a, 0x28, 0x67, 0x69, 0x74,
0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x73, 0x61, 0x66, 0x65, 0x64, 0x65, 0x70, 0x2f,
0x76, 0x65, 0x74, 0x2f, 0x67, 0x65, 0x6e, 0x2f, 0x65, 0x78, 0x63, 0x65, 0x70, 0x74, 0x69, 0x6f,
0x6e, 0x73, 0x61, 0x70, 0x69, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
file_exceptions_spec_proto_rawDescOnce sync.Once
file_exceptions_spec_proto_rawDescData = file_exceptions_spec_proto_rawDesc
)
func file_exceptions_spec_proto_rawDescGZIP() []byte {
file_exceptions_spec_proto_rawDescOnce.Do(func() {
file_exceptions_spec_proto_rawDescData = protoimpl.X.CompressGZIP(file_exceptions_spec_proto_rawDescData)
})
return file_exceptions_spec_proto_rawDescData
}
var file_exceptions_spec_proto_msgTypes = make([]protoimpl.MessageInfo, 2)
var file_exceptions_spec_proto_goTypes = []interface{}{
(*Exception)(nil), // 0: Exception
(*ExceptionSuite)(nil), // 1: ExceptionSuite
}
var file_exceptions_spec_proto_depIdxs = []int32{
0, // 0: ExceptionSuite.exceptions:type_name -> Exception
1, // [1:1] is the sub-list for method output_type
1, // [1:1] is the sub-list for method input_type
1, // [1:1] is the sub-list for extension type_name
1, // [1:1] is the sub-list for extension extendee
0, // [0:1] is the sub-list for field type_name
}
func init() { file_exceptions_spec_proto_init() }
func file_exceptions_spec_proto_init() {
if File_exceptions_spec_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_exceptions_spec_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Exception); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_exceptions_spec_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*ExceptionSuite); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_exceptions_spec_proto_rawDesc,
NumEnums: 0,
NumMessages: 2,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_exceptions_spec_proto_goTypes,
DependencyIndexes: file_exceptions_spec_proto_depIdxs,
MessageInfos: file_exceptions_spec_proto_msgTypes,
}.Build()
File_exceptions_spec_proto = out.File
file_exceptions_spec_proto_rawDesc = nil
file_exceptions_spec_proto_goTypes = nil
file_exceptions_spec_proto_depIdxs = nil
}

2
go.mod
View File

@ -8,7 +8,7 @@ require (
github.com/google/cel-go v0.13.0 github.com/google/cel-go v0.13.0
github.com/google/osv-scanner v1.1.0 github.com/google/osv-scanner v1.1.0
github.com/jedib0t/go-pretty/v6 v6.4.4 github.com/jedib0t/go-pretty/v6 v6.4.4
github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55 github.com/safedep/dry v0.0.0-20230222132026-c8b6cb976849
github.com/sirupsen/logrus v1.9.0 github.com/sirupsen/logrus v1.9.0
github.com/spf13/cobra v1.6.1 github.com/spf13/cobra v1.6.1
github.com/stretchr/testify v1.8.1 github.com/stretchr/testify v1.8.1

2
go.sum
View File

@ -48,6 +48,8 @@ github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJ
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55 h1:OBzggSWzjyEa7YaXp2DvpKDe1wYXtOEcFXQfDqkK7PI= github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55 h1:OBzggSWzjyEa7YaXp2DvpKDe1wYXtOEcFXQfDqkK7PI=
github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55/go.mod h1:odFOtG1l46k23IaCY6kdNkkLW8L+NT+EUVYYVphP59I= github.com/safedep/dry v0.0.0-20230218045153-1a93b0397b55/go.mod h1:odFOtG1l46k23IaCY6kdNkkLW8L+NT+EUVYYVphP59I=
github.com/safedep/dry v0.0.0-20230222132026-c8b6cb976849 h1:5nO9ht1jn7XHFyNFRhUneDZbKmwh4kRr0w/EoWuOQQA=
github.com/safedep/dry v0.0.0-20230222132026-c8b6cb976849/go.mod h1:odFOtG1l46k23IaCY6kdNkkLW8L+NT+EUVYYVphP59I=
github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0= github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0=
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA= github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA=

24
main.go
View File

@ -9,13 +9,15 @@ import (
"github.com/safedep/dry/utils" "github.com/safedep/dry/utils"
"github.com/safedep/vet/internal/ui" "github.com/safedep/vet/internal/ui"
"github.com/safedep/vet/pkg/common/logger" "github.com/safedep/vet/pkg/common/logger"
"github.com/safedep/vet/pkg/exceptions"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
var ( var (
verbose bool verbose bool
debug bool debug bool
logFile string logFile string
globalExceptionsFile string
) )
var banner string = ` var banner string = `
@ -50,6 +52,7 @@ func main() {
cmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "Show verbose logs") cmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "Show verbose logs")
cmd.PersistentFlags().BoolVarP(&debug, "debug", "d", false, "Show debug logs") cmd.PersistentFlags().BoolVarP(&debug, "debug", "d", false, "Show debug logs")
cmd.PersistentFlags().StringVarP(&logFile, "log", "l", "", "Write command logs to file") cmd.PersistentFlags().StringVarP(&logFile, "log", "l", "", "Write command logs to file")
cmd.PersistentFlags().StringVarP(&globalExceptionsFile, "exceptions", "e", "", "Load exceptions from file")
cmd.AddCommand(newAuthCommand()) cmd.AddCommand(newAuthCommand())
cmd.AddCommand(newScanCommand()) cmd.AddCommand(newScanCommand())
@ -58,6 +61,7 @@ func main() {
cobra.OnInitialize(func() { cobra.OnInitialize(func() {
printBanner() printBanner()
loadExceptions()
logger.SetLogLevel(verbose, debug) logger.SetLogLevel(verbose, debug)
}) })
@ -66,6 +70,20 @@ func main() {
} }
} }
func loadExceptions() {
if globalExceptionsFile != "" {
loader, err := exceptions.NewExceptionsFileLoader(globalExceptionsFile)
if err != nil {
logger.Fatalf("Exceptions loader: %v", err)
}
err = exceptions.Load(loader)
if err != nil {
logger.Fatalf("Exceptions loader: %v", err)
}
}
}
func printBanner() { func printBanner() {
bRet, err := strconv.ParseBool(os.Getenv("VET_DISABLE_BANNER")) bRet, err := strconv.ParseBool(os.Getenv("VET_DISABLE_BANNER"))
if (err != nil) || (!bRet) { if (err != nil) || (!bRet) {

View File

@ -9,6 +9,7 @@ import (
"github.com/safedep/vet/pkg/analyzer/filter" "github.com/safedep/vet/pkg/analyzer/filter"
"github.com/safedep/vet/pkg/common/logger" "github.com/safedep/vet/pkg/common/logger"
"github.com/safedep/vet/pkg/models" "github.com/safedep/vet/pkg/models"
"github.com/safedep/vet/pkg/readers"
) )
type celFilterAnalyzer struct { type celFilterAnalyzer struct {
@ -48,7 +49,7 @@ func (f *celFilterAnalyzer) Analyze(manifest *models.PackageManifest,
logger.Infof("CEL filtering manifest: %s", manifest.Path) logger.Infof("CEL filtering manifest: %s", manifest.Path)
f.stat.IncScannedManifest() f.stat.IncScannedManifest()
for _, pkg := range manifest.Packages { readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
f.stat.IncEvaluatedPackage() f.stat.IncEvaluatedPackage()
res, err := f.evaluator.EvalPackage(pkg) res, err := f.evaluator.EvalPackage(pkg)
@ -59,19 +60,21 @@ func (f *celFilterAnalyzer) Analyze(manifest *models.PackageManifest,
pkg.PackageDetails.Name, pkg.PackageDetails.Name,
pkg.PackageDetails.Version, err) pkg.PackageDetails.Version, err)
continue return nil
} }
if res.Matched() { if res.Matched() {
// Avoid duplicates added to the table // Avoid duplicates added to the table
if _, ok := f.packages[pkg.Id()]; ok { if _, ok := f.packages[pkg.Id()]; ok {
continue return nil
} }
f.stat.IncMatchedPackage() f.stat.IncMatchedPackage()
f.packages[pkg.Id()] = pkg f.packages[pkg.Id()] = pkg
} }
}
return nil
})
return f.notifyCaller(manifest, handler) return f.notifyCaller(manifest, handler)
} }

View File

@ -10,6 +10,7 @@ import (
"github.com/safedep/vet/pkg/analyzer/filter" "github.com/safedep/vet/pkg/analyzer/filter"
"github.com/safedep/vet/pkg/common/logger" "github.com/safedep/vet/pkg/common/logger"
"github.com/safedep/vet/pkg/models" "github.com/safedep/vet/pkg/models"
"github.com/safedep/vet/pkg/readers"
) )
type celFilterMatchedPackage struct { type celFilterMatchedPackage struct {
@ -62,7 +63,7 @@ func (f *celFilterSuiteAnalyzer) Analyze(manifest *models.PackageManifest,
logger.Infof("CEL Filter Suite: Analyzing manifest: %s", manifest.Path) logger.Infof("CEL Filter Suite: Analyzing manifest: %s", manifest.Path)
f.stat.IncScannedManifest() f.stat.IncScannedManifest()
for _, pkg := range manifest.Packages { readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
f.stat.IncEvaluatedPackage() f.stat.IncEvaluatedPackage()
res, err := f.evaluator.EvalPackage(pkg) res, err := f.evaluator.EvalPackage(pkg)
@ -73,13 +74,15 @@ func (f *celFilterSuiteAnalyzer) Analyze(manifest *models.PackageManifest,
pkg.PackageDetails.Name, pkg.PackageDetails.Name,
pkg.PackageDetails.Version, err) pkg.PackageDetails.Version, err)
continue return nil
} }
if res.Matched() { if res.Matched() {
f.queueMatchedPkg(pkg, res.GetMatchedFilter().Name()) f.queueMatchedPkg(pkg, res.GetMatchedFilter().Name())
} }
}
return nil
})
if f.failOnMatch && (len(f.matchedPackages) > 0) { if f.failOnMatch && (len(f.matchedPackages) > 0) {
handler(&AnalyzerEvent{ handler(&AnalyzerEvent{

View File

@ -0,0 +1,115 @@
package analyzer
import (
"io"
"os"
"time"
"github.com/safedep/dry/utils"
"github.com/safedep/vet/gen/exceptionsapi"
"github.com/safedep/vet/pkg/analyzer/filter"
"github.com/safedep/vet/pkg/common/logger"
"github.com/safedep/vet/pkg/models"
"github.com/safedep/vet/pkg/readers"
)
type exceptionsGenerator struct {
writer io.WriteCloser
filterEvaluator filter.Evaluator
expires time.Time
pkgCache map[string]*models.Package
}
type ExceptionsGeneratorConfig struct {
Path string
Filter string
ExpiresOn string
}
func NewExceptionsGenerator(config ExceptionsGeneratorConfig) (Analyzer, error) {
fd, err := os.Create(config.Path)
if err != nil {
return nil, err
}
expiresOn, err := time.Parse("2006-01-02", config.ExpiresOn)
if err != nil {
return nil, err
}
filterEvaluator, err := filter.NewEvaluator("exceptions-generator", true)
if err != nil {
return nil, err
}
if utils.IsEmptyString(config.Filter) {
config.Filter = "true"
}
err = filterEvaluator.AddFilter("exceptions-filter", config.Filter)
if err != nil {
return nil, err
}
logger.Infof("Initialized exceptions generator with filter: '%s' expiry: %s",
config.Filter, expiresOn.Format(time.RFC3339))
return &exceptionsGenerator{
writer: fd,
filterEvaluator: filterEvaluator,
expires: expiresOn,
pkgCache: make(map[string]*models.Package),
}, nil
}
func (f *exceptionsGenerator) Name() string {
return "Exceptions Generator"
}
func (f *exceptionsGenerator) Analyze(manifest *models.PackageManifest,
handler AnalyzerEventHandler) error {
readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
res, err := f.filterEvaluator.EvalPackage(pkg)
if err != nil {
return err
}
if !res.Matched() {
return nil
}
f.pkgCache[pkg.Id()] = pkg
return nil
})
return nil
}
func (f *exceptionsGenerator) Finish() error {
defer f.writer.Close()
suite := exceptionsapi.ExceptionSuite{
Name: "Auto Generated Exceptions",
Description: "Exceptions file auto-generated using vet",
Exceptions: make([]*exceptionsapi.Exception, 0),
}
for _, pkg := range f.pkgCache {
logger.Infof("Adding %s to exceptions list", pkg.ShortName())
suite.Exceptions = append(suite.Exceptions, &exceptionsapi.Exception{
Id: utils.NewUniqueId(),
Ecosystem: string(pkg.Ecosystem),
Name: pkg.Name,
Version: pkg.Version,
Expires: f.expires.Format(time.RFC3339),
})
}
err := utils.FromPbToYaml(f.writer, &suite)
if err != nil {
return err
}
return nil
}

View File

@ -51,6 +51,10 @@ func Debugf(msg string, args ...any) {
logrus.Debugf(msg, args...) logrus.Debugf(msg, args...)
} }
func Fatalf(msg string, args ...any) {
logrus.Fatalf(msg, args...)
}
func LoggerWith(key string, value any) *logrus.Entry { func LoggerWith(key string, value any) *logrus.Entry {
return logrus.WithFields(logrus.Fields{ return logrus.WithFields(logrus.Fields{
key: value, key: value,

View File

@ -0,0 +1,148 @@
package exceptions
import (
"fmt"
"hash/fnv"
"io"
"strconv"
"strings"
"sync"
"time"
"github.com/safedep/vet/gen/exceptionsapi"
"github.com/safedep/vet/pkg/models"
)
var (
jitter = 5 * time.Second
)
// Represents an exception rule as per spec with additional details
type exceptionRule struct {
spec *exceptionsapi.Exception
expiry time.Time
}
// In-memory store of exceptions to be used for package hash and exception ID
// based lookup for fast matching and avoid duplicates
type exceptionStore struct {
m sync.RWMutex
rules map[string]map[string]*exceptionRule
}
// Represents an exceptions loader interface to support loading exceptions
// from multiple sources
type exceptionsLoader interface {
// Read an exception rule, return io.EOF on done
Read() (*exceptionRule, error)
}
// Represents an exception match result
type exceptionMatchResult struct {
pkg *models.Package
rule *exceptionRule
}
// Global exceptions store
var globalExceptions *exceptionStore
// Initialize the global exceptions cache
func init() {
initStore()
}
func initStore() {
globalExceptions = &exceptionStore{
rules: make(map[string]map[string]*exceptionRule),
}
}
func Load(loader exceptionsLoader) error {
globalExceptions.m.Lock()
defer globalExceptions.m.Unlock()
for {
rule, err := loader.Read()
if err != nil {
if err == io.EOF {
break
} else {
return err
}
}
if rule.expiry.Before(time.Now().Add(jitter)) {
continue
}
h := pkgHash(rule.spec.GetEcosystem(), rule.spec.GetName())
if _, ok := globalExceptions.rules[h]; ok {
if _, ok = globalExceptions.rules[h][rule.spec.GetId()]; ok {
continue
}
} else {
globalExceptions.rules[h] = make(map[string]*exceptionRule)
}
globalExceptions.rules[h][rule.spec.GetId()] = rule
}
return nil
}
func Apply(pkg *models.Package) (*exceptionMatchResult, error) {
return globalExceptions.Match(pkg)
}
func ActiveCount() int {
return globalExceptions.ActiveCount()
}
func (s *exceptionStore) ActiveCount() int {
return len(s.rules)
}
func (s *exceptionStore) Match(pkg *models.Package) (*exceptionMatchResult, error) {
result := exceptionMatchResult{}
s.m.RLock()
defer s.m.RUnlock()
h := pkgHash(string(pkg.PackageDetails.Ecosystem), pkg.PackageDetails.Name)
if _, ok := s.rules[h]; !ok {
return &result, nil
}
for _, rule := range s.rules[h] {
if rule.matchByPattern(pkg) || rule.matchByVersion(pkg) {
result.pkg = pkg
result.rule = rule
return &result, nil
}
}
return &result, nil
}
func (r *exceptionRule) matchByPattern(pkg *models.Package) bool {
return false
}
func (r *exceptionRule) matchByVersion(pkg *models.Package) bool {
return strings.EqualFold(string(pkg.PackageDetails.Ecosystem), r.spec.GetEcosystem()) &&
strings.EqualFold(pkg.PackageDetails.Name, r.spec.GetName()) &&
((r.spec.GetVersion() == "*") || (r.spec.GetVersion() == pkg.PackageDetails.Version))
}
func (r *exceptionMatchResult) Matched() bool {
return (r == nil) || ((r.pkg != nil) && (r.rule != nil))
}
func pkgHash(ecosystem, name string) string {
h := fnv.New64a()
h.Write([]byte(fmt.Sprintf("%s/%s",
strings.ToLower(ecosystem), strings.ToLower(name))))
return strconv.FormatUint(h.Sum64(), 16)
}

View File

@ -0,0 +1,60 @@
package exceptions
import (
"io"
"os"
"sync"
"time"
"github.com/safedep/dry/utils"
"github.com/safedep/vet/gen/exceptionsapi"
)
type exceptionsFileLoader struct {
m sync.Mutex
idx int
suite *exceptionsapi.ExceptionSuite
}
func NewExceptionsFileLoader(path string) (exceptionsLoader, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
return newExceptionsFileLoaderUsingReader(file)
}
func newExceptionsFileLoaderUsingReader(reader io.Reader) (exceptionsLoader, error) {
var suite exceptionsapi.ExceptionSuite
err := utils.FromYamlToPb(reader, &suite)
if err != nil {
return nil, err
}
return &exceptionsFileLoader{
suite: &suite,
}, nil
}
func (f *exceptionsFileLoader) Read() (*exceptionRule, error) {
f.m.Lock()
defer f.m.Unlock()
if f.idx >= len(f.suite.Exceptions) {
return nil, io.EOF
}
cx := f.suite.Exceptions[f.idx]
f.idx += 1
expiry, err := time.Parse(time.RFC3339, cx.Expires)
if err != nil {
return nil, err
}
return &exceptionRule{
spec: cx,
expiry: expiry,
}, nil
}

View File

@ -0,0 +1,86 @@
package exceptions
import (
"testing"
"github.com/safedep/vet/pkg/models"
"github.com/stretchr/testify/assert"
)
func TestNewExceptionsFileLoader(t *testing.T) {
cases := []struct {
name string
file string
ecosystem string
pkgName string
version string
match bool
errMsg string
}{
{
"Load a file and match",
"fixtures/1_valid.yml",
"maven",
"p1",
"v1",
true,
"",
},
{
"Match any version",
"fixtures/1_valid.yml",
"maven",
"p2",
"v5-anything",
true,
"",
},
{
"Does not match expired exceptions",
"fixtures/1_valid.yml",
"maven",
"p3",
"v5-anything",
false,
"",
},
{
"Error loading file that does not exist",
"fixtures/1_does_not_exists.yml",
"",
"",
"",
false,
"no such file or directory",
},
{
"Error loading invalid YAML file",
"fixtures/2_invalid.yml",
"",
"",
"",
false,
"unknown field",
},
}
for _, test := range cases {
t.Run(test.name, func(t *testing.T) {
initStore()
loader, err := NewExceptionsFileLoader(test.file)
if test.errMsg != "" {
assert.NotNil(t, err)
assert.ErrorContains(t, err, test.errMsg)
return
}
Load(loader)
pd := models.NewPackageDetail(test.ecosystem, test.pkgName, test.version)
res, _ := Apply(&models.Package{PackageDetails: pd})
assert.Equal(t, test.match, res.Matched())
})
}
}

View File

@ -0,0 +1,252 @@
package exceptions
import (
"io"
"testing"
"time"
"github.com/safedep/vet/gen/exceptionsapi"
"github.com/safedep/vet/pkg/models"
"github.com/stretchr/testify/assert"
)
type exceptionsLoaderMocker struct {
rules []exceptionRule
idx int
}
func (m *exceptionsLoaderMocker) Read() (*exceptionRule, error) {
if m.idx >= len(m.rules) {
return nil, io.EOF
}
r := m.rules[m.idx]
m.idx += 1
return &r, nil
}
func TestLoad(t *testing.T) {
cases := []struct {
name string
rules []exceptionRule
pCount int // Package count
}{
{
"Load a rule",
[]exceptionRule{
{
spec: &exceptionsapi.Exception{
Id: "a",
Ecosystem: "maven",
Name: "p1",
},
expiry: time.Now().Add(1 * time.Hour),
},
},
1,
},
{
"Load two rule for same package",
[]exceptionRule{
{
spec: &exceptionsapi.Exception{
Id: "a",
Ecosystem: "maven",
Name: "p1",
},
expiry: time.Now().Add(1 * time.Hour),
},
{
spec: &exceptionsapi.Exception{
Id: "b",
Ecosystem: "maven",
Name: "p1",
},
expiry: time.Now().Add(1 * time.Hour),
},
},
1,
},
{
"Load two rule for two package",
[]exceptionRule{
{
spec: &exceptionsapi.Exception{
Id: "a",
Ecosystem: "maven",
Name: "p1",
},
expiry: time.Now().Add(1 * time.Hour),
},
{
spec: &exceptionsapi.Exception{
Id: "b",
Ecosystem: "maven",
Name: "p2",
},
expiry: time.Now().Add(1 * time.Hour),
},
},
2,
},
{
"Load an expired rule",
[]exceptionRule{
{
spec: &exceptionsapi.Exception{
Id: "a",
Ecosystem: "maven",
Name: "p1",
},
expiry: time.Now().Add(-1 * time.Hour),
},
},
0,
},
}
for _, test := range cases {
t.Run(test.name, func(t *testing.T) {
// Reset the store
initStore()
Load(&exceptionsLoaderMocker{
rules: test.rules,
})
assert.Equal(t, test.pCount, len(globalExceptions.rules))
})
}
}
func TestApply(t *testing.T) {
cases := []struct {
name string
rules []exceptionRule
ecosystem string
pkgName string
version string
match bool
errMsg string
}{
{
"Match by version",
[]exceptionRule{
{
spec: &exceptionsapi.Exception{
Id: "a",
Ecosystem: "maven",
Name: "p1",
Version: "v1",
},
expiry: time.Now().Add(1 * time.Hour),
},
},
"maven",
"p1",
"v1",
true,
"",
},
{
"Match any version",
[]exceptionRule{
{
spec: &exceptionsapi.Exception{
Id: "a",
Ecosystem: "maven",
Name: "p1",
Version: "*",
},
expiry: time.Now().Add(1 * time.Hour),
},
},
"maven",
"p1",
"v-anything",
true,
"",
},
{
"No Match without a version",
[]exceptionRule{
{
spec: &exceptionsapi.Exception{
Id: "a",
Ecosystem: "maven",
Name: "p1",
Version: "",
},
expiry: time.Now().Add(1 * time.Hour),
},
},
"maven",
"p1",
"v-anything",
false,
"",
},
{
"Match with case-insensitive",
[]exceptionRule{
{
spec: &exceptionsapi.Exception{
Id: "a",
Ecosystem: "MAVEN",
Name: "P1",
Version: "v1",
},
expiry: time.Now().Add(1 * time.Hour),
},
},
"maven",
"p1",
"v1",
true,
"",
},
{
"No match with case-insensitive version",
[]exceptionRule{
{
spec: &exceptionsapi.Exception{
Id: "a",
Ecosystem: "MAVEN",
Name: "P1",
Version: "V1",
},
expiry: time.Now().Add(1 * time.Hour),
},
},
"maven",
"p1",
"v1",
false,
"",
},
}
for _, test := range cases {
t.Run(test.name, func(t *testing.T) {
initStore()
Load(&exceptionsLoaderMocker{
rules: test.rules,
})
pd := models.NewPackageDetail(test.ecosystem, test.pkgName, test.version)
res, err := Apply(&models.Package{
PackageDetails: pd,
})
if test.errMsg != "" {
assert.NotNil(t, err)
assert.ErrorContains(t, err, test.errMsg)
} else {
assert.Nil(t, err)
assert.Equal(t, test.match, res.Matched())
}
})
}
}

View File

@ -0,0 +1,17 @@
name: Exceptions File for Fixture
description: |
Test exceptions file
exceptions:
- ecosystem: maven
name: p1
version: v1
expires: 2050-11-10T23:00:00Z
- ecosystem: maven
name: p2
version: '*'
expires: 2050-11-10T23:00:00Z
- ecosystem: maven
name: p3
version: '*'
expires: 2020-11-10T23:00:00Z

View File

@ -0,0 +1,2 @@
A: 1
B: 2

33
pkg/exceptions/utils.go Normal file
View File

@ -0,0 +1,33 @@
package exceptions
import (
"github.com/safedep/vet/pkg/common/logger"
"github.com/safedep/vet/pkg/models"
)
// AllowedPackages iterates over packages in the manifest and call handler
// only for packages not in the exempted by exception rules
func AllowedPackages(manifest *models.PackageManifest,
handler func(pkg *models.Package) error) error {
for _, pkg := range manifest.Packages {
res, err := Apply(pkg)
if err != nil {
logger.Errorf("Failed to evaluate exception for %s: %v",
pkg.ShortName(), err)
continue
}
if res.Matched() {
logger.Debugf("Ignoring package:%s due to exception rule",
pkg.ShortName())
continue
}
err = handler(pkg)
if err != nil {
return err
}
}
return nil
}

View File

@ -75,6 +75,12 @@ func (p *Package) Id() string {
return strconv.FormatUint(h.Sum64(), 16) return strconv.FormatUint(h.Sum64(), 16)
} }
func (p *Package) ShortName() string {
return fmt.Sprintf("pkg:%s/%s@%s",
strings.ToLower(string(p.Ecosystem)),
strings.ToLower(p.Name), p.Version)
}
func NewPackageDetail(e, n, v string) lockfile.PackageDetails { func NewPackageDetail(e, n, v string) lockfile.PackageDetails {
return lockfile.PackageDetails{ return lockfile.PackageDetails{
Ecosystem: lockfile.Ecosystem(e), Ecosystem: lockfile.Ecosystem(e),

18
pkg/readers/pm.go Normal file
View File

@ -0,0 +1,18 @@
package readers
import (
"github.com/safedep/vet/pkg/exceptions"
"github.com/safedep/vet/pkg/models"
)
type packageManifestReader struct {
manifest *models.PackageManifest
}
func NewManifestModelReader(manifest *models.PackageManifest) PackageReader {
return &packageManifestReader{manifest: manifest}
}
func (r *packageManifestReader) EnumPackages(handler func(pkg *models.Package) error) error {
return exceptions.AllowedPackages(r.manifest, handler)
}

16
pkg/readers/reader.go Normal file
View File

@ -0,0 +1,16 @@
package readers
import "github.com/safedep/vet/pkg/models"
// Contract for implementing package manifest readers such as lockfile parser,
// SBOM parser etc. Reader should stop enumeration and return error if handler
// returns an error
type PackageManifestReader interface {
EnumManifests(func(*models.PackageManifest) error) error
}
// Contract for implementing a package reader. Enumerator should fail and return
// error if handler fails
type PackageReader interface {
EnumPackages(func(*models.Package) error) error
}

View File

@ -11,6 +11,7 @@ import (
"github.com/safedep/vet/pkg/analyzer" "github.com/safedep/vet/pkg/analyzer"
"github.com/safedep/vet/pkg/models" "github.com/safedep/vet/pkg/models"
"github.com/safedep/vet/pkg/policy" "github.com/safedep/vet/pkg/policy"
"github.com/safedep/vet/pkg/readers"
) )
type consoleReporter struct{} type consoleReporter struct{}
@ -29,9 +30,10 @@ func (r *consoleReporter) AddManifest(manifest *models.PackageManifest) {
tbl.SetStyle(table.StyleLight) tbl.SetStyle(table.StyleLight)
tbl.AppendHeader(table.Row{"Package", "Attribute", "Summary"}) tbl.AppendHeader(table.Row{"Package", "Attribute", "Summary"})
for _, pkg := range manifest.Packages { readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
r.report(tbl, pkg) r.report(tbl, pkg)
} return nil
})
fmt.Print(text.Bold.Sprint("Manifest: ", text.FgBlue.Sprint(manifest.Path))) fmt.Print(text.Bold.Sprint("Manifest: ", text.FgBlue.Sprint(manifest.Path)))
fmt.Print("\n") fmt.Print("\n")

View File

@ -12,8 +12,10 @@ import (
"github.com/safedep/dry/utils" "github.com/safedep/dry/utils"
"github.com/safedep/vet/gen/insightapi" "github.com/safedep/vet/gen/insightapi"
"github.com/safedep/vet/pkg/analyzer" "github.com/safedep/vet/pkg/analyzer"
"github.com/safedep/vet/pkg/exceptions"
"github.com/safedep/vet/pkg/models" "github.com/safedep/vet/pkg/models"
"github.com/safedep/vet/pkg/policy" "github.com/safedep/vet/pkg/policy"
"github.com/safedep/vet/pkg/readers"
) )
const ( const (
@ -72,13 +74,14 @@ func (r *summaryReporter) Name() string {
} }
func (r *summaryReporter) AddManifest(manifest *models.PackageManifest) { func (r *summaryReporter) AddManifest(manifest *models.PackageManifest) {
for _, pkg := range manifest.Packages { readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
r.processForVulns(pkg) r.processForVulns(pkg)
r.processForPopularity(pkg) r.processForPopularity(pkg)
r.processForVersionDrift(pkg) r.processForVersionDrift(pkg)
r.summary.packages += 1 r.summary.packages += 1
} return nil
})
r.summary.manifests += 1 r.summary.manifests += 1
} }
@ -196,9 +199,14 @@ func (r *summaryReporter) Finish() error {
r.renderRemediationAdvice() r.renderRemediationAdvice()
fmt.Println() fmt.Println()
if exceptions.ActiveCount() > 0 {
fmt.Println(text.Faint.Sprint(summaryListPrependText, r.exceptionsCountStatement()))
fmt.Println()
}
fmt.Println("Run with `vet --filter=\"...\"` for custom filters to identify risky libraries") fmt.Println("Run with `vet --filter=\"...\"` for custom filters to identify risky libraries")
fmt.Println()
fmt.Println("For more details", text.Bold.Sprint("https://github.com/safedep/vet")) fmt.Println("For more details", text.Bold.Sprint("https://github.com/safedep/vet"))
fmt.Println()
return nil return nil
} }
@ -296,3 +304,8 @@ func (r *summaryReporter) majorVersionDriftStatement() string {
return fmt.Sprintf("%d libraries are out of date with major version drift in direct dependencies", return fmt.Sprintf("%d libraries are out of date with major version drift in direct dependencies",
r.summary.metrics.drifts) r.summary.metrics.drifts)
} }
func (r *summaryReporter) exceptionsCountStatement() string {
return fmt.Sprintf("%d libraries are exempted from analysis through exception rules",
exceptions.ActiveCount())
}

View File

@ -7,6 +7,7 @@ import (
"github.com/safedep/vet/pkg/common/logger" "github.com/safedep/vet/pkg/common/logger"
"github.com/safedep/vet/pkg/common/utils" "github.com/safedep/vet/pkg/common/utils"
"github.com/safedep/vet/pkg/models" "github.com/safedep/vet/pkg/models"
"github.com/safedep/vet/pkg/readers"
"github.com/safedep/vet/pkg/reporter" "github.com/safedep/vet/pkg/reporter"
) )
@ -217,9 +218,10 @@ func (s *packageManifestScanner) enrichManifest(manifest *models.PackageManifest
q.Start() q.Start()
for _, pkg := range manifest.Packages { readers.NewManifestModelReader(manifest).EnumPackages(func(pkg *models.Package) error {
q.Add(pkg) q.Add(pkg)
} return nil
})
q.Wait() q.Wait()
q.Stop() q.Stop()

View File

@ -1,6 +1,8 @@
package main package main
import ( import (
"time"
"github.com/safedep/dry/utils" "github.com/safedep/dry/utils"
"github.com/safedep/vet/pkg/analyzer" "github.com/safedep/vet/pkg/analyzer"
"github.com/safedep/vet/pkg/reporter" "github.com/safedep/vet/pkg/reporter"
@ -16,6 +18,11 @@ var (
queryEnableConsoleReport bool queryEnableConsoleReport bool
queryEnableSummaryReport bool queryEnableSummaryReport bool
queryMarkdownReportPath string queryMarkdownReportPath string
queryExceptionsFile string
queryExceptionsTill string
queryExceptionsFilter string
queryDefaultExceptionExpiry = time.Now().Add(90 * 24 * time.Hour)
) )
func newQueryCommand() *cobra.Command { func newQueryCommand() *cobra.Command {
@ -36,6 +43,13 @@ func newQueryCommand() *cobra.Command {
"Filter packages using CEL Filter Suite from file") "Filter packages using CEL Filter Suite from file")
cmd.Flags().BoolVarP(&queryFilterFailOnMatch, "filter-fail", "", false, cmd.Flags().BoolVarP(&queryFilterFailOnMatch, "filter-fail", "", false,
"Fail the command if filter matches any package (for security gate)") "Fail the command if filter matches any package (for security gate)")
cmd.Flags().StringVarP(&queryExceptionsFile, "exceptions-generate", "", "",
"Generate exception records to file (YAML)")
cmd.Flags().StringVarP(&queryExceptionsTill, "exceptions-till", "",
queryDefaultExceptionExpiry.Format("2006-01-02"),
"Generated exceptions are valid till")
cmd.Flags().StringVarP(&queryExceptionsFilter, "exceptions-filter", "", "",
"Generate exception records for packages matching filter")
cmd.Flags().BoolVarP(&queryEnableConsoleReport, "report-console", "", false, cmd.Flags().BoolVarP(&queryEnableConsoleReport, "report-console", "", false,
"Minimal summary of package manifest") "Minimal summary of package manifest")
cmd.Flags().BoolVarP(&queryEnableSummaryReport, "report-summary", "", false, cmd.Flags().BoolVarP(&queryEnableSummaryReport, "report-summary", "", false,
@ -74,6 +88,20 @@ func internalStartQuery() error {
analyzers = append(analyzers, task) analyzers = append(analyzers, task)
} }
if !utils.IsEmptyString(queryExceptionsFile) {
task, err := analyzer.NewExceptionsGenerator(analyzer.ExceptionsGeneratorConfig{
Path: queryExceptionsFile,
ExpiresOn: queryExceptionsTill,
Filter: queryExceptionsFilter,
})
if err != nil {
return err
}
analyzers = append(analyzers, task)
}
if queryEnableConsoleReport { if queryEnableConsoleReport {
rp, err := reporter.NewConsoleReporter() rp, err := reporter.NewConsoleReporter()
if err != nil { if err != nil {

13
samples/exceptions.yml Normal file
View File

@ -0,0 +1,13 @@
name: Sample Exceptions File
description: |
This is a sample exceptions file
exceptions:
- ecosystem: maven
name: com.example.mypackage:mylib1
version: '1.2.3'
expires: 2009-11-10T23:00:00Z
- ecosystem: maven
name: com.example.mypackage:mylib2
version: '*'
expires: 2009-11-10T23:00:00Z