Compare commits

..

15 Commits

Author SHA1 Message Date
Gykes
db4b33f537 Fix: Size Summary Should Represent All Files (#7006)
* modify image size check + tests
* modify scene size check + tests
2026-06-11 20:22:50 +10:00
Gykes
8abbac98d5 New: Support JXL Images in Zips (#7002)
* add extension and update scan logic
* update docs
2026-06-11 20:19:06 +10:00
dependabot[bot]
ed18ea12d4 Bump immutable in /pkg/plugin/examples/react-component (#6946)
Bumps [immutable](https://github.com/immutable-js/immutable-js) from 5.1.4 to 5.1.5.
- [Release notes](https://github.com/immutable-js/immutable-js/releases)
- [Changelog](https://github.com/immutable-js/immutable-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/immutable-js/immutable-js/compare/v5.1.4...v5.1.5)

---
updated-dependencies:
- dependency-name: immutable
  dependency-version: 5.1.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-11 19:52:44 +10:00
Slick Daddy
9cb2ffd56c feat: Add depth support to tag count filters for hierarchical sidebar display (#6929)
* Replace tag query count tests with table-based tests
* Add test for scene count depth = 1
---------
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2026-06-11 15:07:55 +10:00
Slick Daddy
a827006b50 Fix phash precision loss in JSON import/export (#6905)
* Fix phash precision loss in JSON import/export (#6894)
* Use json.RawMessage for Fingerprint. Manually unmarshal numbers into int64.

Reject floating point numbers.

* Add unit test
---------
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2026-06-11 14:07:45 +10:00
Stash-KennyG
a31ccef98d Add sorting and filtering options to sub-groups UI (#6900)
* Add sorting to sub-group UI

- Implemented sorting functionality for sub-groups based on their descriptions in the database queries.
- Updated the GroupList component to allow sorting regardless of the view.
- Added localization for sub-group description in en-GB.json.
- Included sub-group description in the list of sortable options in the groups filter model.

* Implement view filtering and saving.

- Refactor GroupList and GroupSubGroupsPanel components for consistency with general models.
- Removed unused defaultFilter from GroupList and GroupSubGroupsPanel.
- Simplified FilteredGroupList by eliminating unnecessary state variables.
- Fixed dropdown alignment issue in FilteredListToolbar by setting menuPortalTarget to document.body.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Add defaultSort and manualSortBy FilteredGroupList fields.

manualSortBy indicates which sort by value allows manual reordering.

Sets both fields to "sub_group_order" in GroupSubGroupsPanel.

* Revert getSort changes

This function needs to be refactored separately

* Handle sub_group_description without getSort

Eliminates the need to poke group-specific logic into getSort.

---------
Co-authored-by: KennyG <kennyg@kennyg.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2026-06-11 11:08:56 +10:00
Slick Daddy
f76f66b725 feat(studio): add child_ids support to Studio API and UI (#6898)
* Add child_ids to studio update GraphQL schema input
2026-06-10 14:58:24 +10:00
Stash-KennyG
45ca5a4d27 Add hoverPlacementLabel prop to ScrapedPerformersRow component (#6892)
Co-authored-by: KennyG <kennyg@kennyg.com>
2026-06-10 14:45:22 +10:00
Wasylq
763f6903bc [feat] Configure fields when batch tagging performers (#6885)
* Configure fields when batch tagging performers

Added a "Configure fields" button that allows you to select which fields
get refreshed when you do "batch update performers"

This field allows you to sync all data with exceptions, or merging your
local edits with stashbox data
2026-06-10 14:37:51 +10:00
Slick Daddy
759ec317cd Add gallery sorting by performer age (#6879)
* Add gallery sorting by performer age
* Handle null performer ages in gallery sorting
* Document performer age sort semantics and add multi-performer tests
* Fix performer age gallery sorting integration tests.

Import testify/require and replace invalid date helper usage with parsed model dates so integration-tagged sqlite tests compile and run.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix ineffassign in performer_age gallery sort fallback.

Declare fallback before branching so golangci-lint ineffassign passes.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-10 14:21:09 +10:00
Slick Daddy
f3bfd8db75 feat(api): add scene_filter parameter to findDuplicateScenes (#6884)
* feat(schema): add scene_filter to findDuplicateScenes

Expands the findDuplicateScenes query to accept a full SceneFilterType. This enables filtering out specific directories or tags from the duplication matching process.

* feat(backend): update SceneReaderWriter interface to accept filter

Modifies the FindDuplicates signature to take a SceneFilterType pointer, allowing the underlying repository to filter the pool of scenes before duplicate checking.

* test(backend): update SceneReaderWriter mock

Updates the mock definition to match the new FindDuplicates method signature.

* feat(api): pass scene filter to duplicate checking repository

Modifies the FindDuplicateScenes GraphQL resolver to pass the newly added scene_filter schema argument into the database repository layer.

* feat(sqlite): dynamically build FindDuplicates query

Refactors the FindDuplicates implementation to use the internal qb.makeQuery tool instead of static raw SQL. This enables the duplicate checker to utilize the provided SceneFilterType, natively supporting advanced filtering like path exclusions.

* test(sqlite): update duplicate checking tests

Updates the unit tests to pass a nil scene filter, matching the new FindDuplicates method signature.

* test: add comprehensive scene_filter tests for FindDuplicates

- Add TestSceneStore_FindDuplicatesWithFilter with 5 test cases:
  * nil filter baseline
  * tag filter includes only tagged duplicate pair
  * non-existent tag filter returns empty
  * fuzzy matching + tag filter
- Move intPtr helper to setup_test.go
2026-06-10 14:00:25 +10:00
NativeRavenclaw
e470ee34d8 New: Add File Paths to Image Scraper Input (#6856)
* Add Path to image scraper args
* Add a proper imageInput struct for scraper args
* Add test for imageInputFromImage
2026-06-10 13:30:57 +10:00
InfiniteStash
e28ab14c6e Remove phash when oshash changes (#7021) 2026-06-09 09:51:02 +10:00
DogmaDragon
c39e1657ae Replace relative with absolute links 2026-06-07 04:55:52 +03:00
DogmaDragon
58f8b17196 docs: Add gallery and images to built-in Auto Tag scraper desc (#7012) 2026-06-05 08:49:11 -07:00
50 changed files with 1839 additions and 535 deletions

View File

@@ -23,8 +23,8 @@
## Checklist
<!-- Mark [x] to indicate completion. -->
- [ ] I have read and understood the [Contributing](docs/CONTRIBUTING.md) document.
- [ ] I have read and understood the [AI Usage Policy](docs/AI_POLICY.md) document.
- [ ] I have read and understood the [Contributing](https://github.com/stashapp/stash/blob/develop/docs/CONTRIBUTING.md) document.
- [ ] I have read and understood the [AI Usage Policy](https://github.com/stashapp/stash/blob/develop/docs/AI_POLICY.md) document.
- [ ] I have made corresponding changes to the documentation (if applicable).
## AI Usage Disclosure

View File

@@ -51,6 +51,7 @@ type Query {
Fractional seconds are ok: 0.5 will mean only files that have durations within 0.5 seconds between them will be matched based on PHash distance.
"""
duration_diff: Float
scene_filter: SceneFilterType
): [[Scene!]!]!
"Return valid stream paths"

View File

@@ -633,28 +633,28 @@ input TagFilterType {
is_missing: String
"Filter by number of scenes with this tag"
scene_count: IntCriterionInput
scene_count: HierarchicalCountInput
"Filter by number of images with this tag"
image_count: IntCriterionInput
image_count: HierarchicalCountInput
"Filter by number of galleries with this tag"
gallery_count: IntCriterionInput
gallery_count: HierarchicalCountInput
"Filter by number of performers with this tag"
performer_count: IntCriterionInput
performer_count: HierarchicalCountInput
"Filter by number of studios with this tag"
studio_count: IntCriterionInput
studio_count: HierarchicalCountInput
"Filter by number of movies with this tag"
movie_count: IntCriterionInput
movie_count: HierarchicalCountInput
"Filter by number of group with this tag"
group_count: IntCriterionInput
group_count: HierarchicalCountInput
"Filter by number of markers with this tag"
marker_count: IntCriterionInput
marker_count: HierarchicalCountInput
"Filter by parent tags"
parents: HierarchicalMultiCriterionInput
@@ -914,6 +914,14 @@ input IntCriterionInput {
modifier: CriterionModifier!
}
input HierarchicalCountInput {
value: Int!
value2: Int
modifier: CriterionModifier!
"Number of descendant levels to include. Negative values include all descendants."
depth: Int
}
input FloatCriterionInput {
value: Float!
value2: Float

View File

@@ -302,6 +302,8 @@ input StashBoxBatchTagInput {
stash_box_endpoint: String
"Fields to exclude when executing the tagging"
exclude_fields: [String!]
"Collection fields to merge (add to existing) instead of overwriting when executing the tagging"
merge_fields: [String!]
"Refresh items already tagged by StashBox if true. Only tag items with no StashBox tagging if false"
refresh: Boolean!
"If batch adding studios, should their parent studios also be created?"

View File

@@ -36,6 +36,7 @@ input StudioCreateInput {
url: String @deprecated(reason: "Use urls")
urls: [String!]
parent_id: ID
child_ids: [ID!]
"This should be a URL or a base64 encoded data URL"
image: String
stash_ids: [StashIDInput!]
@@ -58,6 +59,7 @@ input StudioUpdateInput {
url: String @deprecated(reason: "Use urls")
urls: [String!]
parent_id: ID
child_ids: [ID!]
"This should be a URL or a base64 encoded data URL"
image: String
stash_ids: [StashIDInput!]

View File

@@ -14,6 +14,63 @@ import (
)
// used to refetch studio after hooks run
func clearRemovedChildStudios(ctx context.Context, qb models.StudioReaderWriter, parentStudioID int, childStudioIDs []int) error {
currentChildren, err := qb.FindChildren(ctx, parentStudioID)
if err != nil {
return err
}
newChildStudioIDs := make(map[int]struct{}, len(childStudioIDs))
for _, childStudioID := range childStudioIDs {
newChildStudioIDs[childStudioID] = struct{}{}
}
for _, currentChild := range currentChildren {
if _, keep := newChildStudioIDs[currentChild.ID]; keep {
continue
}
clearParentPartial := models.NewStudioPartial()
clearParentPartial.ID = currentChild.ID
clearParentPartial.ParentID = models.NewOptionalIntPtr(nil)
if _, err := qb.UpdatePartial(ctx, clearParentPartial); err != nil {
return err
}
}
return nil
}
func setChildStudios(ctx context.Context, qb models.StudioReaderWriter, parentStudioID int, childStudioIDs []int) error {
if err := clearRemovedChildStudios(ctx, qb, parentStudioID, childStudioIDs); err != nil {
return err
}
newChildStudioIDs := make(map[int]struct{}, len(childStudioIDs))
for _, childStudioID := range childStudioIDs {
if _, found := newChildStudioIDs[childStudioID]; found {
continue
}
newChildStudioIDs[childStudioID] = struct{}{}
childPartial := models.NewStudioPartial()
childPartial.ID = childStudioID
childPartial.ParentID = models.NewOptionalInt(parentStudioID)
if err := studio.ValidateModify(ctx, childPartial, qb); err != nil {
return err
}
if _, err := qb.UpdatePartial(ctx, childPartial); err != nil {
return err
}
}
return nil
}
func (r *mutationResolver) getStudio(ctx context.Context, id int) (ret *models.Studio, err error) {
if err := r.withTxn(ctx, func(ctx context.Context) error {
ret, err = r.repository.Studio.Find(ctx, id)
@@ -62,6 +119,11 @@ func (r *mutationResolver) StudioCreate(ctx context.Context, input models.Studio
if err != nil {
return nil, fmt.Errorf("converting tag ids: %w", err)
}
childStudioIDs, err := stringslice.StringSliceToIntSlice(input.ChildIds)
if err != nil {
return nil, fmt.Errorf("converting child ids: %w", err)
}
newStudio.CustomFields = convertMapJSONNumbers(input.CustomFields)
// Process the base 64 encoded image string
@@ -93,6 +155,12 @@ func (r *mutationResolver) StudioCreate(ctx context.Context, input models.Studio
}
}
if input.ChildIds != nil {
if err := setChildStudios(ctx, qb, newStudio.ID, childStudioIDs); err != nil {
return err
}
}
return nil
}); err != nil {
return nil, err
@@ -135,6 +203,11 @@ func (r *mutationResolver) StudioUpdate(ctx context.Context, input models.Studio
return nil, fmt.Errorf("converting tag ids: %w", err)
}
childStudioIDs, err := stringslice.StringSliceToIntSlice(input.ChildIds)
if err != nil {
return nil, fmt.Errorf("converting child ids: %w", err)
}
if translator.hasField("urls") {
// ensure url not included in the input
if err := validateNoLegacyURLs(translator); err != nil {
@@ -197,6 +270,12 @@ func (r *mutationResolver) StudioUpdate(ctx context.Context, input models.Studio
}
}
if translator.hasField("child_ids") {
if err := clearRemovedChildStudios(ctx, qb, studioID, childStudioIDs); err != nil {
return err
}
}
if err := studio.ValidateModify(ctx, updatedStudio, qb); err != nil {
return err
}
@@ -212,6 +291,12 @@ func (r *mutationResolver) StudioUpdate(ctx context.Context, input models.Studio
}
}
if translator.hasField("child_ids") {
if err := setChildStudios(ctx, qb, studioID, childStudioIDs); err != nil {
return err
}
}
return nil
}); err != nil {
return nil, err

View File

@@ -227,7 +227,7 @@ func (r *queryResolver) ParseSceneFilenames(ctx context.Context, filter *models.
return ret, nil
}
func (r *queryResolver) FindDuplicateScenes(ctx context.Context, distance *int, durationDiff *float64) (ret [][]*models.Scene, err error) {
func (r *queryResolver) FindDuplicateScenes(ctx context.Context, distance *int, durationDiff *float64, sceneFilter *models.SceneFilterType) (ret [][]*models.Scene, err error) {
dist := 0
durDiff := -1.
if distance != nil {
@@ -237,7 +237,7 @@ func (r *queryResolver) FindDuplicateScenes(ctx context.Context, distance *int,
durDiff = *durationDiff
}
if err := r.withReadTxn(ctx, func(ctx context.Context) error {
ret, err = r.repository.Scene.FindDuplicates(ctx, dist, durDiff)
ret, err = r.repository.Scene.FindDuplicates(ctx, dist, durDiff, sceneFilter)
return err
}); err != nil {
return nil, err

View File

@@ -315,7 +315,7 @@ const (
// slice default values
var (
defaultVideoExtensions = []string{"m4v", "mp4", "mov", "wmv", "avi", "mpg", "mpeg", "rmvb", "rm", "flv", "asf", "mkv", "webm", "f4v"}
defaultImageExtensions = []string{"png", "jpg", "jpeg", "gif", "webp", "avif"}
defaultImageExtensions = []string{"png", "jpg", "jpeg", "gif", "webp", "avif", "jxl"}
defaultGalleryExtensions = []string{"zip", "cbz"}
defaultMenuItems = []string{"scenes", "images", "groups", "markers", "galleries", "performers", "studios", "tags"}
)

View File

@@ -431,6 +431,8 @@ type StashBoxBatchTagInput struct {
StashBoxEndpoint *string `json:"stash_box_endpoint"`
// Fields to exclude when executing the tagging
ExcludeFields []string `json:"exclude_fields"`
// Collection fields to merge (add to existing) instead of overwriting when executing the tagging
MergeFields []string `json:"merge_fields"`
// Refresh items already tagged by StashBox if true. Only tag items with no StashBox tagging if false
Refresh bool `json:"refresh"`
// If batch adding studios or tags, should their parent entities also be created?
@@ -480,6 +482,7 @@ func (s *Manager) batchTagPerformersByIds(ctx context.Context, input StashBoxBat
performer: performer,
box: box,
excludedFields: input.ExcludeFields,
mergeFields: input.MergeFields,
})
}
}
@@ -500,6 +503,7 @@ func (s *Manager) batchTagPerformersByNamesOrStashIds(input StashBoxBatchTagInpu
stashID: &stashID,
box: box,
excludedFields: input.ExcludeFields,
mergeFields: input.MergeFields,
})
}
}
@@ -516,6 +520,7 @@ func (s *Manager) batchTagPerformersByNamesOrStashIds(input StashBoxBatchTagInpu
name: &name,
box: box,
excludedFields: input.ExcludeFields,
mergeFields: input.MergeFields,
})
}
}
@@ -546,6 +551,7 @@ func (s *Manager) batchTagAllPerformers(ctx context.Context, input StashBoxBatch
performer: performer,
box: box,
excludedFields: input.ExcludeFields,
mergeFields: input.MergeFields,
})
}
return nil

View File

@@ -3,6 +3,7 @@ package manager
import (
"archive/zip"
"context"
gojson "encoding/json"
"fmt"
"io"
"os"
@@ -29,6 +30,7 @@ import (
"github.com/stashapp/stash/pkg/sliceutil/stringslice"
"github.com/stashapp/stash/pkg/studio"
"github.com/stashapp/stash/pkg/tag"
"github.com/stashapp/stash/pkg/utils"
)
type ExportTask struct {
@@ -423,9 +425,25 @@ func fileToJSON(f models.File) jsonschema.DirEntry {
}
for _, fp := range bf.Fingerprints {
fingerprintValue := fp.Fingerprint
// Convert phash to hex string
if fp.Type == models.FingerprintTypePhash {
if v, ok := fp.Fingerprint.(int64); ok {
fingerprintValue = utils.PhashToString(v)
}
}
// encode manually into json.RawMessage
fvEncoded, err := gojson.Marshal(fingerprintValue)
if err != nil {
// ignore - should not happen
logger.Warnf("[files] <%s> error encoding fingerprint %q value: %v", base.Filename(), fp.Type, err)
continue
}
base.Fingerprints = append(base.Fingerprints, jsonschema.Fingerprint{
Type: fp.Type,
Fingerprint: fp.Fingerprint,
Fingerprint: gojson.RawMessage(fvEncoded),
})
}

View File

@@ -27,6 +27,7 @@ type stashBoxBatchPerformerTagTask struct {
stashID *string
performer *models.Performer
excludedFields []string
mergeFields []string
}
func (t *stashBoxBatchPerformerTagTask) getName() string {
@@ -54,8 +55,13 @@ func (t *stashBoxBatchPerformerTagTask) Start(ctx context.Context) {
excluded[field] = true
}
merge := map[string]bool{}
for _, field := range t.mergeFields {
merge[field] = true
}
if performer != nil {
t.processMatchedPerformer(ctx, performer, excluded)
t.processMatchedPerformer(ctx, performer, excluded, merge)
} else {
logger.Infof("No match found for %s", t.getName())
}
@@ -157,7 +163,7 @@ func (t *stashBoxBatchPerformerTagTask) handleMergedPerformer(ctx context.Contex
return mergedPerformer, nil
}
func (t *stashBoxBatchPerformerTagTask) processMatchedPerformer(ctx context.Context, p *models.ScrapedPerformer, excluded map[string]bool) {
func (t *stashBoxBatchPerformerTagTask) processMatchedPerformer(ctx context.Context, p *models.ScrapedPerformer, excluded map[string]bool, merge map[string]bool) {
if t.performer != nil {
storedID, _ := strconv.Atoi(*p.StoredID)
@@ -176,7 +182,7 @@ func (t *stashBoxBatchPerformerTagTask) processMatchedPerformer(ctx context.Cont
return err
}
partial := p.ToPartial(t.box.Endpoint, excluded, existingStashIDs)
partial := p.ToPartial(t.box.Endpoint, excluded, merge, existingStashIDs)
// if we're setting the performer's aliases, and not the name, then filter out the name
// from the aliases to avoid duplicates

View File

@@ -5,6 +5,8 @@ import (
"errors"
"fmt"
"image"
"io"
"os"
"path/filepath"
"strings"
@@ -33,10 +35,18 @@ func (d *Decorator) Decorate(ctx context.Context, fs models.FS, f models.File) (
// ignore clips in non-OsFS filesystems as ffprobe cannot read them
// TODO - copy to temp file if not an OsFS
if _, isOs := fs.(*file.OsFS); !isOs {
ext := strings.ToLower(filepath.Ext(base.Path))
// AVIF images inside zip files are not supported
if strings.ToLower(filepath.Ext(base.Path)) == ".avif" {
if ext == ".avif" {
return nil, fmt.Errorf("%w: %s", ErrUnsupportedAVIFInZip, base.Path)
}
// Go cannot decode JXL from a stream, so extract to a temp file and probe by path
if ext == ".jxl" {
return d.decorateViaTempFile(fs, f)
}
logger.Debugf("assuming ImageFile for non-OsFS file %q", base.Path)
return decorateFallback(fs, f)
}
@@ -137,6 +147,46 @@ func decorateFallback(fs models.FS, f models.File) (models.File, error) {
return ret, nil
}
// decorateViaTempFile extracts a non-OsFS file (e.g. inside a zip) to a temp file so ffprobe can read it by path, for formats like JXL that ffprobe reads but Go cannot decode from a stream.
func (d *Decorator) decorateViaTempFile(fs models.FS, f models.File) (models.File, error) {
base := f.Base()
r, err := fs.Open(base.Path)
if err != nil {
return nil, err
}
defer r.Close()
// preserve the extension so ffprobe can detect the format
tmp, err := os.CreateTemp("", "stash-image-*"+filepath.Ext(base.Path))
if err != nil {
return nil, err
}
defer os.Remove(tmp.Name())
if _, err := io.Copy(tmp, r); err != nil {
tmp.Close()
return nil, err
}
tmp.Close()
probe, err := d.FFProbe.NewVideoFile(tmp.Name())
if err != nil {
return nil, err
}
ret := &models.ImageFile{
BaseFile: base,
Format: probe.VideoCodec,
Width: probe.Width,
Height: probe.Height,
}
adjustForOrientation(fs, base.Path, ret)
return ret, nil
}
func (d *Decorator) IsMissingMetadata(ctx context.Context, fs models.FS, f models.File) bool {
const (
unsetString = "unset"

View File

@@ -2,13 +2,16 @@ package file
import (
"context"
gojson "encoding/json"
"errors"
"fmt"
"path/filepath"
"strings"
"time"
"github.com/stashapp/stash/pkg/models"
"github.com/stashapp/stash/pkg/models/jsonschema"
"github.com/stashapp/stash/pkg/utils"
)
var ErrZipFileNotExist = errors.New("zip file does not exist")
@@ -92,6 +95,31 @@ func (i *Importer) fileJSONToFile(ctx context.Context, fileJSON jsonschema.DirEn
return nil, errors.New("unknown file type")
}
func unmarshalFingerprintValue(fp gojson.RawMessage) (interface{}, error) {
// try to unmarshal as string first
var str string
if err := gojson.Unmarshal(fp, &str); err == nil {
return str, nil
}
// if that fails, try to unmarshal as number
var num gojson.Number
if err := gojson.Unmarshal(fp, &num); err == nil {
// reject floating point values
if strings.Contains(num.String(), ".") {
return nil, fmt.Errorf("floating point fingerprint values are not supported: %s", num.String())
}
ret, err := num.Int64()
if err != nil {
return nil, fmt.Errorf("error parsing fingerprint number: %v", err)
}
return ret, nil
}
return nil, fmt.Errorf("unable to unmarshal fingerprint value: %s", string(fp))
}
func (i *Importer) baseFileJSONToBaseFile(ctx context.Context, baseJSON *jsonschema.BaseFile) (*models.BaseFile, error) {
baseFile := models.BaseFile{
DirEntry: models.DirEntry{
@@ -104,9 +132,32 @@ func (i *Importer) baseFileJSONToBaseFile(ctx context.Context, baseJSON *jsonsch
}
for _, fp := range baseJSON.Fingerprints {
fingerprintValue, err := unmarshalFingerprintValue(fp.Fingerprint)
if err != nil {
return nil, fmt.Errorf("error unmarshaling fingerprint value for type %q: %v", fp.Type, err)
}
// Handle phash: convert hex string to int64, or use legacy int64 values
if fp.Type == models.FingerprintTypePhash {
switch v := fingerprintValue.(type) {
case string:
// New format: hex string
phash, err := utils.StringToPhash(v)
if err != nil {
return nil, fmt.Errorf("error parsing phash hex string %q: %v", v, err)
} else {
fingerprintValue = phash
}
case int64:
// Old format: int64 number
// nothing to do
default:
return nil, fmt.Errorf("unexpected type for phash fingerprint: %T", v)
}
}
baseFile.Fingerprints = append(baseFile.Fingerprints, models.Fingerprint{
Type: fp.Type,
Fingerprint: fp.Fingerprint,
Fingerprint: fingerprintValue,
})
}

137
pkg/file/import_test.go Normal file
View File

@@ -0,0 +1,137 @@
package file
import (
"context"
gojson "encoding/json"
"testing"
"time"
"github.com/stashapp/stash/pkg/models"
"github.com/stashapp/stash/pkg/models/jsonschema"
"github.com/stretchr/testify/assert"
)
func TestImporterPreImport(t *testing.T) {
type testCase struct {
name string
input jsonschema.DirEntry
expected *models.BaseFile
err bool
}
testCases := []testCase{
{
name: "phash as hex string",
input: &jsonschema.BaseFile{
Fingerprints: []jsonschema.Fingerprint{
{
Type: models.FingerprintTypePhash,
Fingerprint: gojson.RawMessage(`"0123456789abcdef"`),
},
},
},
expected: &models.BaseFile{
Fingerprints: []models.Fingerprint{
{
Type: models.FingerprintTypePhash,
Fingerprint: int64(0x0123456789abcdef),
},
},
},
},
{
name: "phash as legacy int64",
input: &jsonschema.BaseFile{
Fingerprints: []jsonschema.Fingerprint{
{
Type: models.FingerprintTypePhash,
Fingerprint: gojson.RawMessage(`1234567890123456789`),
},
},
},
expected: &models.BaseFile{
Fingerprints: []models.Fingerprint{
{
Type: models.FingerprintTypePhash,
Fingerprint: int64(1234567890123456789),
},
},
},
},
{
name: "phash as legacy int64 #6894",
input: &jsonschema.BaseFile{
Fingerprints: []jsonschema.Fingerprint{
{
Type: models.FingerprintTypePhash,
Fingerprint: gojson.RawMessage(`6391326009969271747`),
},
},
},
expected: &models.BaseFile{
Fingerprints: []models.Fingerprint{
{
Type: models.FingerprintTypePhash,
Fingerprint: int64(6391326009969271747),
},
},
},
},
{
name: "floating point fingerprint value",
input: &jsonschema.BaseFile{
Fingerprints: []jsonschema.Fingerprint{
{
Type: models.FingerprintTypeMD5,
Fingerprint: gojson.RawMessage(`12345.6789`),
},
},
},
err: true,
},
{
name: "invalid hex string phash",
input: &jsonschema.BaseFile{
Fingerprints: []jsonschema.Fingerprint{
{
Type: models.FingerprintTypePhash,
Fingerprint: gojson.RawMessage(`"not a hex string"`),
},
},
},
err: true,
},
}
ctx := context.Background()
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
assert := assert.New(t)
importer := &Importer{
Input: tc.input,
}
err := importer.PreImport(ctx)
if !tc.err && err != nil {
t.Fatalf("unexpected error: %v", err)
}
if tc.err {
assert.Error(err)
return
}
// unset timestamps and basename to avoid test failures due to time differences
if importer.file != nil {
importer.file.Base().CreatedAt = time.Time{}
importer.file.Base().UpdatedAt = time.Time{}
importer.file.Base().ModTime = time.Time{}
importer.file.Base().Basename = ""
}
assert.Equal(tc.expected, importer.file)
})
}
}

View File

@@ -843,6 +843,14 @@ func (s *Scanner) removeOutdatedFingerprints(existing models.File, fp models.Fin
return
}
b := existing.Base()
// oshash has changed - drop phash in case file contents are different
if b.Fingerprints.For(models.FingerprintTypePhash) != nil {
logger.Infof("Removing outdated phash from %s", b.Path)
b.Fingerprints = b.Fingerprints.Remove(models.FingerprintTypePhash)
}
md5 := fp.For(models.FingerprintTypeMD5)
if md5 != nil {
@@ -851,8 +859,7 @@ func (s *Scanner) removeOutdatedFingerprints(existing models.File, fp models.Fin
}
// oshash has changed, MD5 is missing - remove MD5 from the existing fingerprints
logger.Infof("Removing outdated checksum from %s", existing.Base().Path)
b := existing.Base()
logger.Infof("Removing outdated checksum from %s", b.Path)
b.Fingerprints = b.Fingerprints.Remove(models.FingerprintTypeMD5)
}

View File

@@ -130,6 +130,21 @@ func (i IntCriterionInput) ValidModifier() bool {
return false
}
type HierarchicalCountInput struct {
Value int `json:"value"`
Value2 *int `json:"value2"`
Modifier CriterionModifier `json:"modifier"`
Depth *int `json:"depth"`
}
func (i HierarchicalCountInput) ValidModifier() bool {
switch i.Modifier {
case CriterionModifierEquals, CriterionModifierNotEquals, CriterionModifierGreaterThan, CriterionModifierLessThan, CriterionModifierIsNull, CriterionModifierNotNull, CriterionModifierBetween, CriterionModifierNotBetween:
return true
}
return false
}
type FloatCriterionInput struct {
Value float64 `json:"value"`
Value2 *float64 `json:"value2"`

View File

@@ -2,6 +2,7 @@ package jsonschema
import (
"bytes"
gojson "encoding/json"
"errors"
"fmt"
"io"
@@ -71,8 +72,10 @@ func (f *BaseFile) IsFile() bool {
}
type Fingerprint struct {
Type string `json:"type,omitempty"`
Fingerprint interface{} `json:"fingerprint,omitempty"`
Type string `json:"type,omitempty"`
// Fingerprint can be string or number, so use RawMessage and unmarshal manually
// #6894: if this is interface{}, it is unmarshalled into a float64 which causes precision loss for phash values.
Fingerprint gojson.RawMessage `json:"fingerprint,omitempty"`
}
type VideoFile struct {

View File

@@ -664,13 +664,13 @@ func (_m *SceneReaderWriter) FindByPrimaryFileID(ctx context.Context, fileID mod
return r0, r1
}
// FindDuplicates provides a mock function with given fields: ctx, distance, durationDiff
func (_m *SceneReaderWriter) FindDuplicates(ctx context.Context, distance int, durationDiff float64) ([][]*models.Scene, error) {
ret := _m.Called(ctx, distance, durationDiff)
// FindDuplicates provides a mock function with given fields: ctx, distance, durationDiff, filter
func (_m *SceneReaderWriter) FindDuplicates(ctx context.Context, distance int, durationDiff float64, filter *models.SceneFilterType) ([][]*models.Scene, error) {
ret := _m.Called(ctx, distance, durationDiff, filter)
var r0 [][]*models.Scene
if rf, ok := ret.Get(0).(func(context.Context, int, float64) [][]*models.Scene); ok {
r0 = rf(ctx, distance, durationDiff)
if rf, ok := ret.Get(0).(func(context.Context, int, float64, *models.SceneFilterType) [][]*models.Scene); ok {
r0 = rf(ctx, distance, durationDiff, filter)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([][]*models.Scene)
@@ -678,8 +678,8 @@ func (_m *SceneReaderWriter) FindDuplicates(ctx context.Context, distance int, d
}
var r1 error
if rf, ok := ret.Get(1).(func(context.Context, int, float64) error); ok {
r1 = rf(ctx, distance, durationDiff)
if rf, ok := ret.Get(1).(func(context.Context, int, float64, *models.SceneFilterType) error); ok {
r1 = rf(ctx, distance, durationDiff, filter)
} else {
r1 = ret.Error(1)
}

View File

@@ -348,13 +348,17 @@ func (p *ScrapedPerformer) GetImage(ctx context.Context, excluded map[string]boo
return nil, nil
}
func (p *ScrapedPerformer) ToPartial(endpoint string, excluded map[string]bool, existingStashIDs []StashID) PerformerPartial {
func (p *ScrapedPerformer) ToPartial(endpoint string, excluded map[string]bool, merge map[string]bool, existingStashIDs []StashID) PerformerPartial {
ret := NewPerformerPartial()
if p.Aliases != nil && !excluded["aliases"] {
mode := RelationshipUpdateModeSet
if merge["aliases"] {
mode = RelationshipUpdateModeAdd
}
ret.Aliases = &UpdateStrings{
Values: stringslice.FromString(*p.Aliases, ","),
Mode: RelationshipUpdateModeSet,
Mode: mode,
}
}
if p.Birthdate != nil && !excluded["birthdate"] {
@@ -430,12 +434,17 @@ func (p *ScrapedPerformer) ToPartial(endpoint string, excluded map[string]bool,
ret.Tattoos = NewOptionalString(*p.Tattoos)
}
urlMode := RelationshipUpdateModeSet
if merge["urls"] {
urlMode = RelationshipUpdateModeAdd
}
// if URLs are provided, only use those
if len(p.URLs) > 0 {
if !excluded["urls"] {
ret.URLs = &UpdateStrings{
Values: p.URLs,
Mode: RelationshipUpdateModeSet,
Mode: urlMode,
}
}
} else {
@@ -453,7 +462,7 @@ func (p *ScrapedPerformer) ToPartial(endpoint string, excluded map[string]bool,
if len(urls) > 0 {
ret.URLs = &UpdateStrings{
Values: urls,
Mode: RelationshipUpdateModeSet,
Mode: urlMode,
}
}
}

View File

@@ -28,7 +28,7 @@ type SceneFinder interface {
FindByPerformerID(ctx context.Context, performerID int) ([]*Scene, error)
FindByGalleryID(ctx context.Context, performerID int) ([]*Scene, error)
FindByGroupID(ctx context.Context, groupID int) ([]*Scene, error)
FindDuplicates(ctx context.Context, distance int, durationDiff float64) ([][]*Scene, error)
FindDuplicates(ctx context.Context, distance int, durationDiff float64, filter *SceneFilterType) ([][]*Scene, error)
}
// SceneQueryer provides methods to query scenes.

View File

@@ -62,6 +62,7 @@ type StudioCreateInput struct {
URL *string `json:"url"` // deprecated
Urls []string `json:"urls"`
ParentID *string `json:"parent_id"`
ChildIds []string `json:"child_ids"`
// This should be a URL or a base64 encoded data URL
Image *string `json:"image"`
StashIds []StashIDInput `json:"stash_ids"`
@@ -82,6 +83,7 @@ type StudioUpdateInput struct {
URL *string `json:"url"` // deprecated
Urls []string `json:"urls"`
ParentID *string `json:"parent_id"`
ChildIds []string `json:"child_ids"`
// This should be a URL or a base64 encoded data URL
Image *string `json:"image"`
StashIds []StashIDInput `json:"stash_ids"`

View File

@@ -15,21 +15,21 @@ type TagFilterType struct {
// Filter to only include tags missing this property
IsMissing *string `json:"is_missing"`
// Filter by number of scenes with this tag
SceneCount *IntCriterionInput `json:"scene_count"`
SceneCount *HierarchicalCountInput `json:"scene_count"`
// Filter by number of images with this tag
ImageCount *IntCriterionInput `json:"image_count"`
ImageCount *HierarchicalCountInput `json:"image_count"`
// Filter by number of galleries with this tag
GalleryCount *IntCriterionInput `json:"gallery_count"`
GalleryCount *HierarchicalCountInput `json:"gallery_count"`
// Filter by number of performers with this tag
PerformerCount *IntCriterionInput `json:"performer_count"`
PerformerCount *HierarchicalCountInput `json:"performer_count"`
// Filter by number of studios with this tag
StudioCount *IntCriterionInput `json:"studio_count"`
StudioCount *HierarchicalCountInput `json:"studio_count"`
// Filter by number of groups with this tag
GroupCount *IntCriterionInput `json:"group_count"`
GroupCount *HierarchicalCountInput `json:"group_count"`
// Filter by number of movies with this tag
MovieCount *IntCriterionInput `json:"movie_count"`
MovieCount *HierarchicalCountInput `json:"movie_count"`
// Filter by number of markers with this tag
MarkerCount *IntCriterionInput `json:"marker_count"`
MarkerCount *HierarchicalCountInput `json:"marker_count"`
// Filter by parent tags
Parents *HierarchicalMultiCriterionInput `json:"parents"`
// Filter by child tags

View File

@@ -9,19 +9,19 @@ importers:
.:
devDependencies:
'@types/react':
specifier: ^18.2.31
specifier: ^18.3.26
version: 18.3.26
'@types/react-dom':
specifier: ^18.2.14
specifier: ^18.3.7
version: 18.3.7(@types/react@18.3.26)
cpx:
specifier: ^1.5.0
version: 1.5.0
sass:
specifier: ^1.69.4
specifier: ^1.93.2
version: 1.93.2
typescript:
specifier: ^5.2.2
specifier: ^5.9.3
version: 5.9.3
packages:
@@ -356,7 +356,7 @@ packages:
glob@7.2.3:
resolution: {integrity: sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q==}
deprecated: Glob versions prior to v9 are no longer supported
deprecated: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me
graceful-fs@4.2.11:
resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==}
@@ -381,8 +381,8 @@ packages:
resolution: {integrity: sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==}
engines: {node: '>= 0.4'}
immutable@5.1.4:
resolution: {integrity: sha512-p6u1bG3YSnINT5RQmx/yRZBpenIl30kVxkTLDyHLIMk0gict704Q9n+thfDI7lTRm9vXdDYutVzXhzcThxTnXA==}
immutable@5.1.5:
resolution: {integrity: sha512-t7xcm2siw+hlUM68I+UEOK+z84RzmN59as9DZ7P1l0994DKUWV7UXBMQZVxaoMSRQ+PBZbHCOoBt7a2wxOMt+A==}
inflight@1.0.6:
resolution: {integrity: sha512-k92I/b08q4wvFscXCLvqfsHCrjrF7yiXsQuIVvVE7N82W3+aqpzuUdBbfhWcy/FZR3/4IgflMgKLOsvPDrGCJA==}
@@ -1160,7 +1160,7 @@ snapshots:
dependencies:
function-bind: 1.1.2
immutable@5.1.4: {}
immutable@5.1.5: {}
inflight@1.0.6:
dependencies:
@@ -1464,7 +1464,7 @@ snapshots:
sass@1.93.2:
dependencies:
chokidar: 4.0.3
immutable: 5.1.4
immutable: 5.1.5
source-map-js: 1.2.1
optionalDependencies:
'@parcel/watcher': 2.5.1

View File

@@ -205,6 +205,52 @@ func galleryInputFromGallery(gallery *models.Gallery) galleryInput {
return ret
}
type imageInput struct {
ID string `json:"id"`
Title string `json:"title"`
Urls []string `json:"urls"`
Date *string `json:"date"`
Details string `json:"details"`
Code string `json:"code,omitempty"`
Photographer string `json:"photographer,omitempty"`
Files []fileInput `json:"files,omitempty"`
}
func imageInputFromImage(image *models.Image) imageInput {
dateToStringPtr := func(s *models.Date) *string {
if s != nil {
v := s.String()
return &v
}
return nil
}
// fallback to file basename if title is empty
title := image.GetTitle()
urls := image.URLs.List()
ret := imageInput{
ID: strconv.Itoa(image.ID),
Title: title,
Urls: urls,
Details: image.Details,
Date: dateToStringPtr(image.Date),
Code: image.Code,
Photographer: image.Photographer,
}
for _, f := range image.Files.List() {
fi := fileInputFromFile(*f.Base())
ret.Files = append(ret.Files, fi)
}
return ret
}
var ErrScraperScript = errors.New("scraper script error")
type scriptScraper struct {
@@ -392,6 +438,9 @@ func (s *scriptFragmentScraper) scrapeByFragment(ctx context.Context, input Inpu
case input.Scene != nil:
inString, err = json.Marshal(*input.Scene)
ty = ScrapeContentTypeScene
case input.Image != nil:
inString, err = json.Marshal(*input.Image)
ty = ScrapeContentTypeImage
}
if err != nil {
@@ -430,7 +479,7 @@ func (s *scriptFragmentScraper) scrapeGalleryByGallery(ctx context.Context, gall
}
func (s *scriptFragmentScraper) scrapeImageByImage(ctx context.Context, image *models.Image) (*models.ScrapedImage, error) {
inString, err := json.Marshal(imageToUpdateInput(image))
inString, err := json.Marshal(imageInputFromImage(image))
if err != nil {
return nil, err

View File

@@ -0,0 +1,52 @@
package scraper
import (
"fmt"
"testing"
"github.com/stashapp/stash/pkg/models"
"github.com/stretchr/testify/assert"
)
func Test_imageInputFromImage_worksWithMultipleFiles(t *testing.T) {
date, _ := models.ParseDate("2020-01-01")
model := models.Image{
ID: 1,
Title: "Test Image",
URLs: models.NewRelatedStrings([]string{"https://example.com/image.png"}),
Date: &date,
Code: "Code",
Photographer: "Photographer",
Files: models.NewRelatedFiles([]models.File{
makeImageFile(1),
makeImageFile(2),
}),
}
input := imageInputFromImage(&model)
assert.Equal(t, "1", input.ID)
assert.Equal(t, "Test Image", input.Title)
assert.Equal(t, "https://example.com/image.png", input.Urls[0])
assert.Equal(t, "2020-01-01", *input.Date)
assert.Equal(t, "Code", input.Code)
assert.Equal(t, "Photographer", input.Photographer)
assert.Equal(t, "/data/images/image_0001_.png", input.Files[0].Path)
assert.Equal(t, "/data/images/image_0002_.png", input.Files[1].Path)
}
func getImageStringValue(index int, field string) string {
return fmt.Sprintf("image_%04d_%s", index, field)
}
func makeImageFile(i int) *models.ImageFile {
return &models.ImageFile{
BaseFile: &models.BaseFile{
Path: "/data/images/" + getImageStringValue(i, ".png"),
Basename: getImageStringValue(i, ".png"),
},
Height: 200,
Width: 300,
}
}

View File

@@ -438,26 +438,3 @@ func (s *stashScraper) scrapeImageByImage(ctx context.Context, image *models.Ima
func (s *stashScraper) scrapeByURL(_ context.Context, _ string, _ ScrapeContentType) (ScrapedContent, error) {
return nil, ErrNotSupported
}
func imageToUpdateInput(gallery *models.Image) models.ImageUpdateInput {
dateToStringPtr := func(s *models.Date) *string {
if s != nil {
v := s.String()
return &v
}
return nil
}
// fallback to file basename if title is empty
title := gallery.GetTitle()
urls := gallery.URLs.List()
return models.ImageUpdateInput{
ID: strconv.Itoa(gallery.ID),
Title: &title,
Details: &gallery.Details,
Urls: urls,
Date: dateToStringPtr(gallery.Date),
}
}

View File

@@ -829,6 +829,7 @@ var gallerySortOptions = sortOptions{
"id",
"images_count",
"path",
"performer_age",
"performer_count",
"random",
"rating",
@@ -890,6 +891,34 @@ func (qb *GalleryStore) setGallerySort(query *queryBuilder, findFilter *models.F
query.sortAndPagination += getCountSort(galleryTable, galleriesTagsTable, galleryIDColumn, direction)
case "performer_count":
query.sortAndPagination += getCountSort(galleryTable, performersGalleriesTable, galleryIDColumn, direction)
case "performer_age":
// Multi-performer semantics:
// - ASC sorts by the youngest performer in each gallery (MIN age)
// - DESC sorts by the oldest performer in each gallery (MAX age)
aggregation := "MIN"
if direction == "DESC" {
// DESC uses oldest performer age for each gallery.
aggregation = "MAX"
}
var fallback string
if direction == "ASC" {
// ASC puts NULL first by default, so coalesce to sqlite max int.
fallback = "9223372036854775807"
} else {
// DESC puts larger values first; coalesce NULL to sqlite min int to keep NULLs last.
fallback = "-9223372036854775808"
}
query.sortAndPagination += fmt.Sprintf(
" ORDER BY (SELECT COALESCE(%s(JulianDay(galleries.date) - JulianDay(performers.birthdate)), %s) FROM %s as performers INNER JOIN %s AS aggregation WHERE performers.id = aggregation.%s AND aggregation.%s = %s.id) %s",
aggregation,
fallback,
performerTable,
performersGalleriesTable,
performerIDColumn,
galleryIDColumn,
galleryTable,
getSortDirection(direction),
)
case "path":
// special handling for path
addFileTable()

View File

@@ -12,6 +12,7 @@ import (
"github.com/stashapp/stash/pkg/models"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
var invalidID = -1
@@ -1504,6 +1505,7 @@ func galleryQueryQ(ctx context.Context, t *testing.T, q string, expectedGalleryI
// no Q should return all results
filter.Q = nil
filter.PerPage = ptr(-1)
galleries, _, err = qb.Query(ctx, nil, &filter)
if err != nil {
t.Errorf("Error querying gallery: %s", err.Error())
@@ -2825,6 +2827,20 @@ func TestGalleryQuerySorting(t *testing.T) {
-1,
-1,
},
{
"performer age asc",
"performer_age",
models.SortDirectionEnumAsc,
-1,
-1,
},
{
"performer age desc",
"performer_age",
models.SortDirectionEnumDesc,
-1,
-1,
},
}
qb := db.Gallery
@@ -2862,6 +2878,163 @@ func TestGalleryQuerySorting(t *testing.T) {
}
}
func TestGalleryQuerySortingPerformerAgeNullHandling(t *testing.T) {
runWithRollbackTxn(t, "performer age null handling", func(t *testing.T, ctx context.Context) {
assert := assert.New(t)
knownBirthdate, err := models.ParseDate("1990-01-01")
require.NoError(t, err)
galleryDate, err := models.ParseDate("2020-01-01")
require.NoError(t, err)
knownPerformer := models.Performer{
Name: "performer-known-birthdate",
Birthdate: &knownBirthdate,
}
require.NoError(t, db.Performer.Create(ctx, &models.CreatePerformerInput{Performer: &knownPerformer}))
unknownPerformer := models.Performer{
Name: "performer-unknown-birthdate",
}
require.NoError(t, db.Performer.Create(ctx, &models.CreatePerformerInput{Performer: &unknownPerformer}))
knownOnlyGallery := models.Gallery{
Title: "gallery-known-only",
Date: &galleryDate,
PerformerIDs: models.NewRelatedIDs([]int{
knownPerformer.ID,
}),
}
require.NoError(t, db.Gallery.Create(ctx, &models.CreateGalleryInput{Gallery: &knownOnlyGallery}))
mixedGallery := models.Gallery{
Title: "gallery-known-and-unknown",
Date: &galleryDate,
PerformerIDs: models.NewRelatedIDs([]int{
knownPerformer.ID,
unknownPerformer.ID,
}),
}
require.NoError(t, db.Gallery.Create(ctx, &models.CreateGalleryInput{Gallery: &mixedGallery}))
unknownOnlyGallery := models.Gallery{
Title: "gallery-unknown-only",
Date: &galleryDate,
PerformerIDs: models.NewRelatedIDs([]int{
unknownPerformer.ID,
}),
}
require.NoError(t, db.Gallery.Create(ctx, &models.CreateGalleryInput{Gallery: &unknownOnlyGallery}))
findIndex := func(galleries []*models.Gallery, id int) int {
for i, g := range galleries {
if g.ID == id {
return i
}
}
return -1
}
asc := models.SortDirectionEnumAsc
sortBy := "performer_age"
ascGot, _, err := db.Gallery.Query(ctx, nil, &models.FindFilterType{Sort: &sortBy, Direction: &asc})
require.NoError(t, err)
ascKnownOnly := findIndex(ascGot, knownOnlyGallery.ID)
ascMixed := findIndex(ascGot, mixedGallery.ID)
ascUnknownOnly := findIndex(ascGot, unknownOnlyGallery.ID)
assert.NotEqual(-1, ascKnownOnly)
assert.NotEqual(-1, ascMixed)
assert.NotEqual(-1, ascUnknownOnly)
assert.Less(ascKnownOnly, ascUnknownOnly)
assert.Less(ascMixed, ascUnknownOnly)
desc := models.SortDirectionEnumDesc
descGot, _, err := db.Gallery.Query(ctx, nil, &models.FindFilterType{Sort: &sortBy, Direction: &desc})
require.NoError(t, err)
descKnownOnly := findIndex(descGot, knownOnlyGallery.ID)
descMixed := findIndex(descGot, mixedGallery.ID)
descUnknownOnly := findIndex(descGot, unknownOnlyGallery.ID)
assert.NotEqual(-1, descKnownOnly)
assert.NotEqual(-1, descMixed)
assert.NotEqual(-1, descUnknownOnly)
assert.Less(descKnownOnly, descUnknownOnly)
assert.Less(descMixed, descUnknownOnly)
})
}
func TestGalleryQuerySortingPerformerAgeMultiPerformerAggregation(t *testing.T) {
runWithRollbackTxn(t, "performer age multi performer aggregation", func(t *testing.T, ctx context.Context) {
assert := assert.New(t)
youngBirthdate, err := models.ParseDate("2000-01-01")
require.NoError(t, err)
midBirthdate, err := models.ParseDate("1990-01-01")
require.NoError(t, err)
oldBirthdate, err := models.ParseDate("1980-01-01")
require.NoError(t, err)
galleryDate, err := models.ParseDate("2020-01-01")
require.NoError(t, err)
young := models.Performer{Name: "performer-young", Birthdate: &youngBirthdate}
mid := models.Performer{Name: "performer-mid", Birthdate: &midBirthdate}
old := models.Performer{Name: "performer-old", Birthdate: &oldBirthdate}
require.NoError(t, db.Performer.Create(ctx, &models.CreatePerformerInput{Performer: &young}))
require.NoError(t, db.Performer.Create(ctx, &models.CreatePerformerInput{Performer: &mid}))
require.NoError(t, db.Performer.Create(ctx, &models.CreatePerformerInput{Performer: &old}))
galleryYoungAndOld := models.Gallery{
Title: "gallery-young-and-old",
Date: &galleryDate,
PerformerIDs: models.NewRelatedIDs([]int{
young.ID,
old.ID,
}),
}
require.NoError(t, db.Gallery.Create(ctx, &models.CreateGalleryInput{Gallery: &galleryYoungAndOld}))
galleryMidOnly := models.Gallery{
Title: "gallery-mid-only",
Date: &galleryDate,
PerformerIDs: models.NewRelatedIDs([]int{
mid.ID,
}),
}
require.NoError(t, db.Gallery.Create(ctx, &models.CreateGalleryInput{Gallery: &galleryMidOnly}))
findIndex := func(galleries []*models.Gallery, id int) int {
for i, g := range galleries {
if g.ID == id {
return i
}
}
return -1
}
sortBy := "performer_age"
asc := models.SortDirectionEnumAsc
ascGot, _, err := db.Gallery.Query(ctx, nil, &models.FindFilterType{Sort: &sortBy, Direction: &asc})
require.NoError(t, err)
ascYoungAndOld := findIndex(ascGot, galleryYoungAndOld.ID)
ascMidOnly := findIndex(ascGot, galleryMidOnly.ID)
assert.NotEqual(-1, ascYoungAndOld)
assert.NotEqual(-1, ascMidOnly)
// ASC uses MIN(age), so gallery with youngest performer should come first.
assert.Less(ascYoungAndOld, ascMidOnly)
desc := models.SortDirectionEnumDesc
descGot, _, err := db.Gallery.Query(ctx, nil, &models.FindFilterType{Sort: &sortBy, Direction: &desc})
require.NoError(t, err)
descYoungAndOld := findIndex(descGot, galleryYoungAndOld.ID)
descMidOnly := findIndex(descGot, galleryMidOnly.ID)
assert.NotEqual(-1, descYoungAndOld)
assert.NotEqual(-1, descMidOnly)
// DESC uses MAX(age), so gallery with oldest performer should come first.
assert.Less(descYoungAndOld, descMidOnly)
})
}
func TestGalleryStore_AddImages(t *testing.T) {
tests := []struct {
name string

View File

@@ -500,6 +500,7 @@ var groupSortOptions = sortOptions{
"rating",
"scenes_count",
"o_counter",
"sub_group_description",
"sub_group_order",
"tag_count",
"updated_at",
@@ -532,6 +533,15 @@ func (qb *GroupStore) setGroupSort(query *queryBuilder, findFilter *models.FindF
query.joinSort(groupRelationsTable, "", "groups.id = groups_relations.sub_id")
query.sortAndPagination += getSort("order_index", direction, groupRelationsTable)
}
case "sub_group_description":
// as above, we need to handle parent groups differently here
const clause = " ORDER BY COALESCE(%s.description, '') COLLATE NATURAL_CI %s"
if query.hasJoin("groups_parents") {
query.sortAndPagination += fmt.Sprintf(clause, "groups_parents", direction)
} else {
query.joinSort(groupRelationsTable, "", "groups.id = groups_relations.sub_id")
query.sortAndPagination += fmt.Sprintf(clause, groupRelationsTable, direction)
}
case "tag_count":
query.sortAndPagination += getCountSort(groupTable, groupsTagsTable, groupIDColumn, direction)
case "scenes_count": // generic getSort won't work for this

View File

@@ -1124,6 +1124,90 @@ func TestGroupQuerySortOrderIndex(t *testing.T) {
})
}
func TestGroupQuerySortSubGroupDescription(t *testing.T) {
runWithRollbackTxn(t, "sort subgroup description", func(t *testing.T, ctx context.Context) {
assert := assert.New(t)
cEmpty := models.Group{Name: "sort-desc-child-empty"}
c01 := models.Group{Name: "sort-desc-child-01"}
c2 := models.Group{Name: "sort-desc-child-2"}
c10 := models.Group{Name: "sort-desc-child-10"}
assert.NoError(db.Group.Create(ctx, &cEmpty))
assert.NoError(db.Group.Create(ctx, &c01))
assert.NoError(db.Group.Create(ctx, &c2))
assert.NoError(db.Group.Create(ctx, &c10))
parent := models.Group{
Name: "sort-desc-parent",
SubGroups: models.NewRelatedGroupDescriptions([]models.GroupIDDescription{
{GroupID: cEmpty.ID, Description: ""},
{GroupID: c10.ID, Description: "10"},
{GroupID: c2.ID, Description: "2"},
{GroupID: c01.ID, Description: "01"},
}),
}
assert.NoError(db.Group.Create(ctx, &parent))
sortKey := "sub_group_description"
dirAsc := models.SortDirectionEnumAsc
findFilter := models.FindFilterType{
Sort: &sortKey,
Direction: &dirAsc,
}
groupFilter := models.GroupFilterType{
ContainingGroups: &models.HierarchicalMultiCriterionInput{
Value: []string{strconv.Itoa(parent.ID)},
Modifier: models.CriterionModifierIncludes,
},
}
groups, _, err := db.Group.Query(ctx, &groupFilter, &findFilter)
assert.NoError(err)
assert.Len(groups, 4)
assert.Equal(cEmpty.ID, groups[0].ID)
assert.Equal(c01.ID, groups[1].ID)
assert.Equal(c2.ID, groups[2].ID)
assert.Equal(c10.ID, groups[3].ID)
dirDesc := models.SortDirectionEnumDesc
findFilter.Direction = &dirDesc
groups, _, err = db.Group.Query(ctx, &groupFilter, &findFilter)
assert.NoError(err)
assert.Len(groups, 4)
assert.Equal(c10.ID, groups[0].ID)
assert.Equal(c2.ID, groups[1].ID)
assert.Equal(c01.ID, groups[2].ID)
assert.Equal(cEmpty.ID, groups[3].ID)
// Exercise the non-groups_parents code path by filtering on name only.
nameCriterion := models.StringCriterionInput{
Value: "sort-desc-child-",
Modifier: models.CriterionModifierIncludes,
}
nameFilter := models.GroupFilterType{
Name: &nameCriterion,
}
findFilter.Direction = &dirAsc
groups, _, err = db.Group.Query(ctx, &nameFilter, &findFilter)
assert.NoError(err)
assert.Len(groups, 4)
assert.Equal(cEmpty.ID, groups[0].ID)
assert.Equal(c01.ID, groups[1].ID)
assert.Equal(c2.ID, groups[2].ID)
assert.Equal(c10.ID, groups[3].ID)
findFilter.Direction = &dirDesc
groups, _, err = db.Group.Query(ctx, &nameFilter, &findFilter)
assert.NoError(err)
assert.Len(groups, 4)
assert.Equal(c10.ID, groups[0].ID)
assert.Equal(c2.ID, groups[1].ID)
assert.Equal(c01.ID, groups[2].ID)
assert.Equal(cEmpty.ID, groups[3].ID)
})
}
func TestGroupUpdateFrontImage(t *testing.T) {
if err := withRollbackTxn(func(ctx context.Context) error {
qb := db.Group

View File

@@ -950,6 +950,11 @@ func (qb *ImageStore) queryGroupedFields(ctx context.Context, options models.Ima
aggregateQuery.addColumn("SUM(temp.size) as size")
}
// #5503 - select the file id so equal-sized/megapixel files aren't collapsed by DISTINCT
if options.Megapixels || options.TotalSize {
query.addColumn(imagesFilesTable + ".file_id")
}
const includeSortPagination = false
aggregateQuery.from = fmt.Sprintf("(%s) as temp", query.toSQL(includeSortPagination))

View File

@@ -1630,6 +1630,67 @@ func queryImagesWithCount(ctx context.Context, sqb models.ImageReader, imageFilt
return images, result.Count, nil
}
// #5503 - total size/megapixels must include secondary files of equal size/dimensions
func TestImageQueryTotalSizeMultipleFiles(t *testing.T) {
withRollbackTxn(func(ctx context.Context) error {
iqb := db.Image
fqb := db.File
const fileSize = int64(1234)
const fileWidth = 1000
const fileHeight = 1000
makeFile := func(basename string) models.FileID {
f := &models.ImageFile{
BaseFile: &models.BaseFile{
Path: getFilePath(folderIdxWithImageFiles, basename),
Basename: basename,
ParentFolderID: folderIDs[folderIdxWithImageFiles],
Size: fileSize,
},
Width: fileWidth,
Height: fileHeight,
}
if err := fqb.Create(ctx, f); err != nil {
t.Fatalf("creating file: %v", err)
}
return f.ID
}
f1 := makeFile("multifile-image-1.jpg")
f2 := makeFile("multifile-image-2.jpg")
image := &models.Image{Title: "multifile image"}
if err := iqb.Create(ctx, &models.CreateImageInput{
Image: image,
FileIDs: []models.FileID{f1, f2},
}); err != nil {
t.Fatalf("creating image: %v", err)
}
result, err := iqb.Query(ctx, models.ImageQueryOptions{
QueryOptions: models.QueryOptions{Count: true},
ImageFilter: &models.ImageFilterType{
ID: &models.IntCriterionInput{
Modifier: models.CriterionModifierEquals,
Value: image.ID,
},
},
Megapixels: true,
TotalSize: true,
})
if err != nil {
t.Fatalf("querying image: %v", err)
}
assert.Equal(t, 1, result.Count)
assert.Equal(t, float64(fileSize*2), result.TotalSize)
assert.Equal(t, float64(fileWidth*fileHeight*2)/1000000, result.Megapixels)
return nil
})
}
func imageQueryQ(ctx context.Context, t *testing.T, sqb models.ImageReader, q string, expectedImageIdx int) {
filter := models.FindFilterType{
Q: &q,

View File

@@ -41,41 +41,6 @@ const (
sceneCoverBlobColumn = "cover_blob"
)
var findExactDuplicateQuery = `
SELECT GROUP_CONCAT(DISTINCT scene_id) as ids
FROM (
SELECT scenes.id as scene_id
, video_files.duration as file_duration
, files.size as file_size
, files_fingerprints.fingerprint as phash
, abs(max(video_files.duration) OVER (PARTITION by files_fingerprints.fingerprint) - video_files.duration) as durationDiff
FROM scenes
INNER JOIN scenes_files ON (scenes.id = scenes_files.scene_id)
INNER JOIN files ON (scenes_files.file_id = files.id)
INNER JOIN files_fingerprints ON (scenes_files.file_id = files_fingerprints.file_id AND files_fingerprints.type = 'phash')
INNER JOIN video_files ON (files.id == video_files.file_id)
)
WHERE durationDiff <= ?1
OR ?1 < 0 -- Always TRUE if the parameter is negative.
-- That will disable the durationDiff checking.
GROUP BY phash
HAVING COUNT(phash) > 1
AND COUNT(DISTINCT scene_id) > 1
ORDER BY SUM(file_size) DESC;
`
var findAllPhashesQuery = `
SELECT scenes.id as id
, files_fingerprints.fingerprint as phash
, video_files.duration as duration
FROM scenes
INNER JOIN scenes_files ON (scenes.id = scenes_files.scene_id)
INNER JOIN files ON (scenes_files.file_id = files.id)
INNER JOIN files_fingerprints ON (scenes_files.file_id = files_fingerprints.file_id AND files_fingerprints.type = 'phash')
INNER JOIN video_files ON (files.id == video_files.file_id)
ORDER BY files.size DESC;
`
type sceneRow struct {
ID int `db:"id" goqu:"skipinsert"`
Title zero.String `db:"title"`
@@ -1121,6 +1086,11 @@ func (qb *SceneStore) queryGroupedFields(ctx context.Context, options models.Sce
aggregateQuery.addColumn("SUM(temp.size) as size")
}
// #5503 - select the file id so equal-sized/duration files aren't collapsed by DISTINCT
if options.TotalDuration || options.TotalSize {
query.addColumn(scenesFilesTable + ".file_id")
}
const includeSortPagination = false
aggregateQuery.from = fmt.Sprintf("(%s) as temp", query.toSQL(includeSortPagination))
@@ -1462,11 +1432,61 @@ func (qb *SceneStore) GetStashIDs(ctx context.Context, sceneID int) ([]models.St
return sceneRepository.stashIDs.get(ctx, sceneID)
}
func (qb *SceneStore) FindDuplicates(ctx context.Context, distance int, durationDiff float64) ([][]*models.Scene, error) {
func (qb *SceneStore) FindDuplicates(ctx context.Context, distance int, durationDiff float64, filter *models.SceneFilterType) ([][]*models.Scene, error) {
var dupeIds [][]int
query, err := qb.makeQuery(ctx, filter, nil)
if err != nil {
return nil, err
}
// Add necessary joins for duplicate checking
query.addJoins(
join{
table: scenesFilesTable,
onClause: "scenes.id = scenes_files.scene_id",
},
join{
table: fileTable,
onClause: "scenes_files.file_id = files.id",
},
join{
table: fingerprintTable,
onClause: "scenes_files.file_id = files_fingerprints.file_id AND files_fingerprints.type = 'phash'",
},
join{
table: videoFileTable,
onClause: "files.id = video_files.file_id",
},
)
if distance == 0 {
query.columns = []string{
"scenes.id as scene_id",
"video_files.duration as file_duration",
"files.size as file_size",
"files_fingerprints.fingerprint as phash",
"abs(max(video_files.duration) OVER (PARTITION by files_fingerprints.fingerprint) - video_files.duration) as durationDiff",
}
sqlStr := query.toSQL(false)
finalQuery := `
SELECT GROUP_CONCAT(DISTINCT scene_id) as ids
FROM (` + sqlStr + `)
WHERE phash IS NOT NULL
AND (durationDiff <= ?
OR ? < 0) -- Always TRUE if the parameter is negative.
-- That will disable the durationDiff checking.
GROUP BY phash
HAVING COUNT(phash) > 1
AND COUNT(DISTINCT scene_id) > 1
ORDER BY SUM(file_size) DESC;
`
var ids []string
if err := dbWrapper.Select(ctx, &ids, findExactDuplicateQuery, durationDiff); err != nil {
args := append(query.allArgs(), durationDiff, durationDiff)
if err := dbWrapper.Select(ctx, &ids, finalQuery, args...); err != nil {
return nil, err
}
@@ -1484,9 +1504,19 @@ func (qb *SceneStore) FindDuplicates(ctx context.Context, distance int, duration
}
}
} else {
query.columns = []string{
"scenes.id as id",
"files_fingerprints.fingerprint as phash",
"video_files.duration as duration",
}
query.addWhere("files_fingerprints.fingerprint IS NOT NULL")
query.sortAndPagination = " ORDER BY files.size DESC"
sqlStr := query.toSQL(true)
var hashes []*utils.Phash
if err := sceneRepository.queryFunc(ctx, findAllPhashesQuery, nil, false, func(rows *sqlx.Rows) error {
if err := sceneRepository.queryFunc(ctx, sqlStr, query.allArgs(), false, func(rows *sqlx.Rows) error {
phash := utils.Phash{
Bucket: -1,
Duration: -1,

View File

@@ -2094,6 +2094,62 @@ func sceneQueryQ(ctx context.Context, t *testing.T, sqb models.SceneReader, q st
assert.Len(t, scenes, totalScenes)
}
// #5503 - total size/duration must include secondary files of equal size/duration
func TestSceneQueryTotalSizeMultipleFiles(t *testing.T) {
withRollbackTxn(func(ctx context.Context) error {
sqb := db.Scene
fqb := db.File
const fileSize = int64(1234)
const fileDuration = float64(100)
makeFile := func(basename string) models.FileID {
f := &models.VideoFile{
BaseFile: &models.BaseFile{
Path: getFilePath(folderIdxWithSceneFiles, basename),
Basename: basename,
ParentFolderID: folderIDs[folderIdxWithSceneFiles],
Size: fileSize,
},
Duration: fileDuration,
}
if err := fqb.Create(ctx, f); err != nil {
t.Fatalf("creating file: %v", err)
}
return f.ID
}
f1 := makeFile("multifile-scene-1.mp4")
f2 := makeFile("multifile-scene-2.mp4")
scene := &models.Scene{Title: "multifile scene"}
if err := sqb.Create(ctx, scene, []models.FileID{f1, f2}); err != nil {
t.Fatalf("creating scene: %v", err)
}
result, err := sqb.Query(ctx, models.SceneQueryOptions{
QueryOptions: models.QueryOptions{Count: true},
SceneFilter: &models.SceneFilterType{
ID: &models.IntCriterionInput{
Modifier: models.CriterionModifierEquals,
Value: scene.ID,
},
},
TotalDuration: true,
TotalSize: true,
})
if err != nil {
t.Fatalf("querying scene: %v", err)
}
assert.Equal(t, 1, result.Count)
assert.Equal(t, float64(fileSize*2), result.TotalSize)
assert.Equal(t, fileDuration*2, result.TotalDuration)
return nil
})
}
func TestSceneQuery(t *testing.T) {
var (
endpoint = sceneStashID(sceneIdxWithGallery).Endpoint
@@ -4162,7 +4218,10 @@ func TestSceneQueryPhashDuplicated(t *testing.T) {
duplicated = false
scenes = queryScene(ctx, t, sqb, &sceneFilter, nil)
findFilter := models.FindFilterType{
PerPage: ptr(-1),
}
scenes = queryScene(ctx, t, sqb, &sceneFilter, &findFilter)
// -1 for missing phash
assert.Len(t, scenes, totalScenes-(dupeScenePhashes*2)-1)
@@ -4631,7 +4690,7 @@ func TestSceneStore_FindDuplicates(t *testing.T) {
withRollbackTxn(func(ctx context.Context) error {
distance := 0
durationDiff := -1.
got, err := qb.FindDuplicates(ctx, distance, durationDiff)
got, err := qb.FindDuplicates(ctx, distance, durationDiff, nil)
if err != nil {
t.Errorf("SceneStore.FindDuplicates() error = %v", err)
return nil
@@ -4641,7 +4700,7 @@ func TestSceneStore_FindDuplicates(t *testing.T) {
distance = 1
durationDiff = -1.
got, err = qb.FindDuplicates(ctx, distance, durationDiff)
got, err = qb.FindDuplicates(ctx, distance, durationDiff, nil)
if err != nil {
t.Errorf("SceneStore.FindDuplicates() error = %v", err)
return nil
@@ -4653,6 +4712,214 @@ func TestSceneStore_FindDuplicates(t *testing.T) {
})
}
func TestSceneStore_FindDuplicatesWithFilter(t *testing.T) {
qb := db.Scene
// Helper to create a scene with a specific phash and optional title prefix
createDupeScene := func(ctx context.Context, name string, phash int64) (*models.Scene, error) {
sceneFile := &models.VideoFile{
BaseFile: &models.BaseFile{
Basename: name,
ParentFolderID: folderIDs[folderIdxWithSceneFiles],
Fingerprints: models.Fingerprints{
{Type: models.FingerprintTypeMD5, Fingerprint: name + "_md5"},
{Type: models.FingerprintTypeOshash, Fingerprint: name + "_oshash"},
{Type: models.FingerprintTypePhash, Fingerprint: phash},
},
},
Duration: 100.0,
Width: 1920,
Height: 1080,
}
if err := db.File.Create(ctx, sceneFile); err != nil {
return nil, err
}
scene := &models.Scene{
Title: name,
}
if err := qb.Create(ctx, scene, []models.FileID{sceneFile.ID}); err != nil {
return nil, err
}
return scene, nil
}
// Helper to add tags to a scene
addSceneTags := func(ctx context.Context, sceneID int, tagIDsToAdd []int) error {
_, err := qb.UpdatePartial(ctx, sceneID, models.ScenePartial{
TagIDs: &models.UpdateIDs{
Mode: models.RelationshipUpdateModeSet,
IDs: tagIDsToAdd,
},
})
return err
}
withRollbackTxn(func(ctx context.Context) error {
// Create a test tag to use for filtering
err := db.Tag.Create(ctx, &models.CreateTagInput{
Tag: &models.Tag{
Name: "FindDuplicatesFilterTestTag",
},
})
if err != nil {
t.Errorf("failed to create test tag: %v", err)
return nil
}
// fetch the tag we just created
tagName := "FindDuplicatesFilterTestTag"
tags, _, err := db.Tag.Query(ctx, &models.TagFilterType{
Name: &models.StringCriterionInput{
Value: tagName,
Modifier: models.CriterionModifierEquals,
},
}, &models.FindFilterType{
PerPage: intPtr(1),
})
if err != nil || len(tags) == 0 {
t.Errorf("failed to find test tag: %v", err)
return nil
}
testTagID := tags[0].ID
// Create two pairs of duplicate scenes:
// Pair A: sceneA1 and sceneA2 have the same phash and share a tag
// Pair B: sceneB1 and sceneB2 have the same phash but no tag
const sharedPhash int64 = 999999
sceneA1, err := createDupeScene(ctx, "FilterTest_A1", sharedPhash)
if err != nil {
t.Errorf("failed to create sceneA1: %v", err)
return nil
}
sceneA2, err := createDupeScene(ctx, "FilterTest_A2", sharedPhash)
if err != nil {
t.Errorf("failed to create sceneA2: %v", err)
return nil
}
const otherPhash int64 = 888888
sceneB1, err := createDupeScene(ctx, "FilterTest_B1", otherPhash)
if err != nil {
t.Errorf("failed to create sceneB1: %v", err)
return nil
}
sceneB2, err := createDupeScene(ctx, "FilterTest_B2", otherPhash)
if err != nil {
t.Errorf("failed to create sceneB2: %v", err)
return nil
}
// Add tag only to pair A
if err := addSceneTags(ctx, sceneA1.ID, []int{testTagID}); err != nil {
t.Errorf("failed to add tag to sceneA1: %v", err)
return nil
}
if err := addSceneTags(ctx, sceneA2.ID, []int{testTagID}); err != nil {
t.Errorf("failed to add tag to sceneA2: %v", err)
return nil
}
// Test 1: No filter - should find all duplicates (2 pairs: original + our new ones)
distance := 0
durationDiff := -1.0
got, err := qb.FindDuplicates(ctx, distance, durationDiff, nil)
if err != nil {
t.Errorf("FindDuplicates(nil filter) error = %v", err)
return nil
}
// Should find at least our 2 new pairs (may find more from pre-populated data)
assert.GreaterOrEqual(t, len(got), 2, "nil filter should find at least our 2 new duplicate pairs")
// Test 2: Filter by tag - should only find pair A (the tagged pair)
got, err = qb.FindDuplicates(ctx, distance, durationDiff, &models.SceneFilterType{
Tags: &models.HierarchicalMultiCriterionInput{
Value: []string{strconv.Itoa(testTagID)},
Modifier: models.CriterionModifierIncludes,
},
})
if err != nil {
t.Errorf("FindDuplicates(tag filter) error = %v", err)
return nil
}
// Should find exactly 1 duplicate pair (pair A)
assert.Len(t, got, 1, "tag filter should find exactly 1 duplicate pair")
// Verify the found pair contains our tagged scenes
if len(got) == 1 {
foundIDs := map[int]bool{}
for _, s := range got[0] {
foundIDs[s.ID] = true
}
assert.True(t, foundIDs[sceneA1.ID], "pair A scene 1 should be in results")
assert.True(t, foundIDs[sceneA2.ID], "pair A scene 2 should be in results")
// Pair B (untagged) should NOT be in the results
assert.False(t, foundIDs[sceneB1.ID], "pair B scene 1 should NOT be in tag-filtered results")
assert.False(t, foundIDs[sceneB2.ID], "pair B scene 2 should NOT be in tag-filtered results")
}
// Test 3: Filter by tag that no duplicate scene has - should find nothing
err = db.Tag.Create(ctx, &models.CreateTagInput{
Tag: &models.Tag{
Name: "FindDuplicatesFilterTestTag_NonExistent",
},
})
if err != nil {
t.Errorf("failed to create non-existent tag: %v", err)
return nil
}
nonExistentTagName := "FindDuplicatesFilterTestTag_NonExistent"
tags2, _, err := db.Tag.Query(ctx, &models.TagFilterType{
Name: &models.StringCriterionInput{
Value: nonExistentTagName,
Modifier: models.CriterionModifierEquals,
},
}, &models.FindFilterType{
PerPage: intPtr(1),
})
if err != nil || len(tags2) == 0 {
t.Errorf("failed to find non-existent tag: %v", err)
return nil
}
nonExistentTagID := tags2[0].ID
got, err = qb.FindDuplicates(ctx, distance, durationDiff, &models.SceneFilterType{
Tags: &models.HierarchicalMultiCriterionInput{
Value: []string{strconv.Itoa(nonExistentTagID)},
Modifier: models.CriterionModifierIncludes,
},
})
if err != nil {
t.Errorf("FindDuplicates(non-existent tag filter) error = %v", err)
return nil
}
assert.Len(t, got, 0, "non-existent tag filter should find no duplicates")
// Test 4: Fuzzy match (distance=1) with filter
distance = 1
got, err = qb.FindDuplicates(ctx, distance, durationDiff, &models.SceneFilterType{
Tags: &models.HierarchicalMultiCriterionInput{
Value: []string{strconv.Itoa(testTagID)},
Modifier: models.CriterionModifierIncludes,
},
})
if err != nil {
t.Errorf("FindDuplicates(fuzzy + tag filter) error = %v", err)
return nil
}
// Should still find pair A with fuzzy matching
assert.Len(t, got, 1, "fuzzy + tag filter should find exactly 1 duplicate pair")
return nil
})
}
func TestSceneStore_AssignFiles(t *testing.T) {
tests := []struct {
name string

View File

@@ -84,6 +84,8 @@ const (
sceneIdxMissingPhash
sceneIdxWithPerformerParentTag
sceneIdxWithGroupWithParent
sceneIdxWithChildTag
sceneIdxWithGrandChildTag
// new indexes above
lastSceneIdx
@@ -115,6 +117,7 @@ const (
imageIdxWithPerformerTwoTags
imageIdxWithGrandChildStudio
imageIdxWithPerformerParentTag
imageIdxWithGrandChildTag
// new indexes above
totalImages
)
@@ -167,6 +170,7 @@ const (
groupIdxWithGrandParent
groupIdxWithParentAndScene
groupIdxWithChildWithScene
groupIdxWithGrandChildTag
// groups with dup names start from the end
groupIdxWithDupName
@@ -199,6 +203,7 @@ const (
galleryIdxWithGrandChildStudio
galleryIdxWithoutFile
galleryIdxWithPerformerParentTag
galleryIdxWithGrandChildTag
// new indexes above
lastGalleryIdx
@@ -378,6 +383,8 @@ var (
sceneIdxWithThreeTags: {tagIdx1WithScene, tagIdx2WithScene, tagIdx3WithScene},
sceneIdxWithMarkerAndTag: {tagIdx3WithScene},
sceneIdxWithMarkerTwoTags: {tagIdx2WithScene, tagIdx3WithScene},
sceneIdxWithChildTag: {tagIdxWithParentTag},
sceneIdxWithGrandChildTag: {tagIdxWithGrandParent},
}
scenePerformers = linkMap{
@@ -430,6 +437,7 @@ var (
{sceneIdxWithMarkers, tagIdxWithPrimaryMarkers, []int{tagIdxWithMarkers, tagIdx2WithMarkers}},
{sceneIdxWithMarkerAndTag, tagIdxWithPrimaryMarkers, nil},
{sceneIdxWithMarkerTwoTags, tagIdxWithPrimaryMarkers, nil},
{sceneIdxWithGrandChildTag, tagIdxWithGrandParent, nil},
}
)
@@ -461,9 +469,10 @@ var (
imageIdxWithGrandChildStudio: studioIdxWithGrandParent,
}
imageTags = linkMap{
imageIdxWithTag: {tagIdxWithImage},
imageIdxWithTwoTags: {tagIdx1WithImage, tagIdx2WithImage},
imageIdxWithThreeTags: {tagIdx1WithImage, tagIdx2WithImage, tagIdx3WithImage},
imageIdxWithTag: {tagIdxWithImage},
imageIdxWithTwoTags: {tagIdx1WithImage, tagIdx2WithImage},
imageIdxWithThreeTags: {tagIdx1WithImage, tagIdx2WithImage, tagIdx3WithImage},
imageIdxWithGrandChildTag: {tagIdxWithGrandParent},
}
imagePerformers = linkMap{
imageIdxWithPerformer: {performerIdxWithImage},
@@ -502,9 +511,10 @@ var (
}
galleryTags = linkMap{
galleryIdxWithTag: {tagIdxWithGallery},
galleryIdxWithTwoTags: {tagIdx1WithGallery, tagIdx2WithGallery},
galleryIdxWithThreeTags: {tagIdx1WithGallery, tagIdx2WithGallery, tagIdx3WithGallery},
galleryIdxWithTag: {tagIdxWithGallery},
galleryIdxWithTwoTags: {tagIdx1WithGallery, tagIdx2WithGallery},
galleryIdxWithThreeTags: {tagIdx1WithGallery, tagIdx2WithGallery, tagIdx3WithGallery},
galleryIdxWithGrandChildTag: {tagIdxWithGrandParent},
}
)
@@ -514,9 +524,10 @@ var (
}
groupTags = linkMap{
groupIdxWithTag: {tagIdxWithGroup},
groupIdxWithTwoTags: {tagIdx1WithGroup, tagIdx2WithGroup},
groupIdxWithThreeTags: {tagIdx1WithGroup, tagIdx2WithGroup, tagIdx3WithGroup},
groupIdxWithTag: {tagIdxWithGroup},
groupIdxWithTwoTags: {tagIdx1WithGroup, tagIdx2WithGroup},
groupIdxWithThreeTags: {tagIdx1WithGroup, tagIdx2WithGroup, tagIdx3WithGroup},
groupIdxWithGrandChildTag: {tagIdxWithGrandParent},
}
)
@@ -588,6 +599,10 @@ func indexesToIDPtrs[T any](ids []T, indexes []int) []*T {
return ret
}
func intPtr(i int) *int {
return &i
}
func indexToIDPtr[T any](ids []T, idx int) *T {
if idx < 0 {
return nil
@@ -605,6 +620,10 @@ func indexFromID(ids []int, id int) int {
return -1
}
func ptr[T any](v T) *T {
return &v
}
var db *sqlite.Database
func TestMain(m *testing.M) {

View File

@@ -88,6 +88,7 @@ func getSortDirection(direction string) string {
return direction
}
}
func getSort(sort string, direction string, tableName string) string {
direction = getSortDirection(direction)

View File

@@ -2,6 +2,7 @@ package sqlite
import (
"context"
"fmt"
"github.com/stashapp/stash/pkg/models"
)
@@ -217,80 +218,112 @@ func (qb *tagFilterHandler) isMissingCriterionHandler(isMissing *string) criteri
}
}
func (qb *tagFilterHandler) sceneCountCriterionHandler(sceneCount *models.IntCriterionInput) criterionHandlerFunc {
return func(ctx context.Context, f *filterBuilder) {
if sceneCount != nil {
f.addLeftJoin("scenes_tags", "", "scenes_tags.tag_id = tags.id")
clause, args := getIntCriterionWhereClause("count(distinct scenes_tags.scene_id)", *sceneCount)
f.addHaving(clause, args...)
}
// addHierarchicalCountCTE adds a recursive CTE to walk the tag hierarchy.
// depth < 0 includes all descendant levels (unlimited recursion).
// depth >= 0 limits the recursion to that many levels from the root tag.
func (qb *tagFilterHandler) addHierarchicalCountCTE(f *filterBuilder, cteAlias string, depth int) {
if depth < 0 {
// unlimited recursion — no depth tracking needed
f.addRecursiveWith(fmt.Sprintf(
`%[1]s(root_id, descendant_id) AS (
SELECT id, id FROM tags
UNION ALL
SELECT td.root_id, tr.child_id
FROM tags_relations tr
INNER JOIN %[1]s td ON td.descendant_id = tr.parent_id
)`, cteAlias))
} else {
// depth-limited: track recursion level as a CTE column
f.addRecursiveWith(fmt.Sprintf(
`%[1]s(root_id, descendant_id, depth) AS (
SELECT id, id, 0 FROM tags
UNION ALL
SELECT td.root_id, tr.child_id, td.depth + 1
FROM tags_relations tr
INNER JOIN %[1]s td ON td.descendant_id = tr.parent_id
WHERE td.depth < %[2]d
)`, cteAlias, depth))
}
}
func (qb *tagFilterHandler) imageCountCriterionHandler(imageCount *models.IntCriterionInput) criterionHandlerFunc {
return func(ctx context.Context, f *filterBuilder) {
if imageCount != nil {
f.addLeftJoin("images_tags", "", "images_tags.tag_id = tags.id")
clause, args := getIntCriterionWhereClause("count(distinct images_tags.image_id)", *imageCount)
func (qb *tagFilterHandler) hierarchicalCountHandler(input *models.HierarchicalCountInput, joinTable, entityIDCol string) criterionHandlerFunc {
return qb.hierarchicalCountHandlerOnCol(input, joinTable, "tag_id", entityIDCol)
}
f.addHaving(clause, args...)
func (qb *tagFilterHandler) hierarchicalCountHandlerOnCol(input *models.HierarchicalCountInput, joinTable, tagCol, entityIDCol string) criterionHandlerFunc {
return func(ctx context.Context, f *filterBuilder) {
if input == nil {
return
}
intInput := models.IntCriterionInput{
Value: input.Value,
Value2: input.Value2,
Modifier: input.Modifier,
}
if input.Depth != nil {
cteAlias := joinTable + "_desc"
qb.addHierarchicalCountCTE(f, cteAlias, *input.Depth)
f.addLeftJoin(cteAlias, "", fmt.Sprintf("%s.root_id = tags.id", cteAlias))
f.addLeftJoin(joinTable, "", fmt.Sprintf("%[1]s.%[2]s = %[3]s.descendant_id", joinTable, tagCol, cteAlias))
} else {
f.addLeftJoin(joinTable, "", fmt.Sprintf("%s.%s = tags.id", joinTable, tagCol))
}
clause, args := getIntCriterionWhereClause(fmt.Sprintf("count(distinct %s.%s)", joinTable, entityIDCol), intInput)
f.addHaving(clause, args...)
}
}
func (qb *tagFilterHandler) galleryCountCriterionHandler(galleryCount *models.IntCriterionInput) criterionHandlerFunc {
return func(ctx context.Context, f *filterBuilder) {
if galleryCount != nil {
f.addLeftJoin("galleries_tags", "", "galleries_tags.tag_id = tags.id")
clause, args := getIntCriterionWhereClause("count(distinct galleries_tags.gallery_id)", *galleryCount)
f.addHaving(clause, args...)
}
}
func (qb *tagFilterHandler) sceneCountCriterionHandler(sceneCount *models.HierarchicalCountInput) criterionHandlerFunc {
return qb.hierarchicalCountHandler(sceneCount, "scenes_tags", "scene_id")
}
func (qb *tagFilterHandler) performerCountCriterionHandler(performerCount *models.IntCriterionInput) criterionHandlerFunc {
return func(ctx context.Context, f *filterBuilder) {
if performerCount != nil {
f.addLeftJoin("performers_tags", "", "performers_tags.tag_id = tags.id")
clause, args := getIntCriterionWhereClause("count(distinct performers_tags.performer_id)", *performerCount)
f.addHaving(clause, args...)
}
}
func (qb *tagFilterHandler) imageCountCriterionHandler(imageCount *models.HierarchicalCountInput) criterionHandlerFunc {
return qb.hierarchicalCountHandler(imageCount, "images_tags", "image_id")
}
func (qb *tagFilterHandler) studioCountCriterionHandler(studioCount *models.IntCriterionInput) criterionHandlerFunc {
return func(ctx context.Context, f *filterBuilder) {
if studioCount != nil {
f.addLeftJoin("studios_tags", "", "studios_tags.tag_id = tags.id")
clause, args := getIntCriterionWhereClause("count(distinct studios_tags.studio_id)", *studioCount)
f.addHaving(clause, args...)
}
}
func (qb *tagFilterHandler) galleryCountCriterionHandler(galleryCount *models.HierarchicalCountInput) criterionHandlerFunc {
return qb.hierarchicalCountHandler(galleryCount, "galleries_tags", "gallery_id")
}
func (qb *tagFilterHandler) groupCountCriterionHandler(groupCount *models.IntCriterionInput) criterionHandlerFunc {
return func(ctx context.Context, f *filterBuilder) {
if groupCount != nil {
f.addLeftJoin("groups_tags", "", "groups_tags.tag_id = tags.id")
clause, args := getIntCriterionWhereClause("count(distinct groups_tags.group_id)", *groupCount)
f.addHaving(clause, args...)
}
}
func (qb *tagFilterHandler) performerCountCriterionHandler(performerCount *models.HierarchicalCountInput) criterionHandlerFunc {
return qb.hierarchicalCountHandler(performerCount, "performers_tags", "performer_id")
}
func (qb *tagFilterHandler) markerCountCriterionHandler(markerCount *models.IntCriterionInput) criterionHandlerFunc {
func (qb *tagFilterHandler) studioCountCriterionHandler(studioCount *models.HierarchicalCountInput) criterionHandlerFunc {
return qb.hierarchicalCountHandler(studioCount, "studios_tags", "studio_id")
}
func (qb *tagFilterHandler) groupCountCriterionHandler(groupCount *models.HierarchicalCountInput) criterionHandlerFunc {
return qb.hierarchicalCountHandler(groupCount, "groups_tags", "group_id")
}
func (qb *tagFilterHandler) markerCountCriterionHandler(markerCount *models.HierarchicalCountInput) criterionHandlerFunc {
return func(ctx context.Context, f *filterBuilder) {
if markerCount != nil {
if markerCount == nil {
return
}
intInput := models.IntCriterionInput{
Value: markerCount.Value,
Value2: markerCount.Value2,
Modifier: markerCount.Modifier,
}
if markerCount.Depth != nil {
cteAlias := "scene_markers_desc"
qb.addHierarchicalCountCTE(f, cteAlias, *markerCount.Depth)
f.addLeftJoin(cteAlias, "", fmt.Sprintf("%s.root_id = tags.id", cteAlias))
f.addLeftJoin("scene_markers_tags", "", fmt.Sprintf("scene_markers_tags.tag_id = %s.descendant_id", cteAlias))
f.addLeftJoin("scene_markers", "", "scene_markers_tags.scene_marker_id = scene_markers.id OR scene_markers.primary_tag_id = "+cteAlias+".descendant_id")
} else {
f.addLeftJoin("scene_markers_tags", "", "scene_markers_tags.tag_id = tags.id")
f.addLeftJoin("scene_markers", "", "scene_markers_tags.scene_marker_id = scene_markers.id OR scene_markers.primary_tag_id = tags.id")
clause, args := getIntCriterionWhereClause("count(distinct scene_markers.id)", *markerCount)
f.addHaving(clause, args...)
}
clause, args := getIntCriterionWhereClause("count(distinct scene_markers.id)", intInput)
f.addHaving(clause, args...)
}
}

View File

@@ -569,318 +569,206 @@ func TestTagQueryIsMissingImage(t *testing.T) {
})
}
func TestTagQuerySceneCount(t *testing.T) {
countCriterion := models.IntCriterionInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
func TestTagQueryCounts(t *testing.T) {
tests := []struct {
name string
buildFilter models.TagFilterType
includeIdxs []int
excludeIdxs []int
}{
{
name: "scene_count_equals_1",
buildFilter: models.TagFilterType{SceneCount: &models.HierarchicalCountInput{Value: 1, Modifier: models.CriterionModifierEquals}},
includeIdxs: []int{tagIdxWithScene},
excludeIdxs: []int{tagIdx1WithNothing, tagIdx1WithScene, tagIdx2WithScene, tagIdx3WithScene},
},
{
name: "scene_count_equals_1_depth_1",
buildFilter: models.TagFilterType{SceneCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(1),
}},
includeIdxs: []int{tagIdxWithScene, tagIdxWithParentTag, tagIdxWithChildTag, tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdx1WithNothing, tagIdx1WithScene, tagIdx2WithScene, tagIdx3WithScene},
},
{
name: "marker_count_equals_2",
buildFilter: models.TagFilterType{MarkerCount: &models.HierarchicalCountInput{Value: 2, Modifier: models.CriterionModifierEquals}},
includeIdxs: []int{tagIdxWithMarkers, tagIdx2WithMarkers},
excludeIdxs: []int{tagIdxWithPrimaryMarkers, tagIdx1WithNothing, tagIdx2WithNothing},
},
{
name: "image_count_equals_1",
buildFilter: models.TagFilterType{ImageCount: &models.HierarchicalCountInput{Value: 1, Modifier: models.CriterionModifierEquals}},
includeIdxs: []int{tagIdxWithImage, tagIdx3WithImage},
excludeIdxs: []int{tagIdx1WithImage, tagIdx2WithImage, tagIdxWithCoverImage, tagIdx1WithNothing, tagIdx2WithNothing},
},
{
name: "gallery_count_equals_1",
buildFilter: models.TagFilterType{GalleryCount: &models.HierarchicalCountInput{Value: 1, Modifier: models.CriterionModifierEquals}},
includeIdxs: []int{tagIdxWithGallery, tagIdx3WithGallery},
excludeIdxs: []int{tagIdx1WithGallery, tagIdx2WithGallery, tagIdx1WithNothing, tagIdx2WithNothing},
},
{
name: "performer_count_equals_1",
buildFilter: models.TagFilterType{PerformerCount: &models.HierarchicalCountInput{Value: 1, Modifier: models.CriterionModifierEquals}},
includeIdxs: []int{tagIdxWithPerformer, tagIdx1WithPerformer, tagIdxWithParentAndChild},
excludeIdxs: []int{tagIdx2WithPerformer, tagIdx1WithNothing, tagIdx2WithNothing},
},
{
name: "studio_count_equals_1",
buildFilter: models.TagFilterType{StudioCount: &models.HierarchicalCountInput{Value: 1, Modifier: models.CriterionModifierEquals}},
includeIdxs: []int{tagIdxWithStudio, tagIdx1WithStudio},
excludeIdxs: []int{tagIdx2WithStudio, tagIdx1WithNothing, tagIdx2WithNothing},
},
{
name: "parent_count_equals_1",
buildFilter: models.TagFilterType{ParentCount: &models.IntCriterionInput{Value: 1, Modifier: models.CriterionModifierEquals}},
includeIdxs: []int{tagIdxWithParentTag, tagIdxWithGrandParent, tagIdxWithParentAndChild},
excludeIdxs: []int{tagIdx1WithNothing, tagIdx2WithNothing},
},
{
name: "child_count_equals_1",
buildFilter: models.TagFilterType{ChildCount: &models.IntCriterionInput{Value: 1, Modifier: models.CriterionModifierEquals}},
includeIdxs: []int{tagIdxWithChildTag, tagIdxWithGrandChild, tagIdxWithParentAndChild},
excludeIdxs: []int{tagIdx1WithNothing, tagIdx2WithNothing},
},
{
name: "scene_count_equals_1_depth_-1",
buildFilter: models.TagFilterType{SceneCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(-1),
}},
includeIdxs: []int{tagIdxWithScene, tagIdxWithChildTag, tagIdxWithParentTag, tagIdxWithGrandChild, tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdx1WithScene, tagIdx2WithScene, tagIdx3WithScene},
},
{
name: "image_count_equals_1_depth_1",
buildFilter: models.TagFilterType{ImageCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(1),
}},
includeIdxs: []int{tagIdxWithImage, tagIdx3WithImage, tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdx1WithImage, tagIdx2WithImage},
},
{
name: "image_count_equals_1_depth_-1",
buildFilter: models.TagFilterType{ImageCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(-1),
}},
includeIdxs: []int{tagIdxWithImage, tagIdx3WithImage, tagIdxWithGrandChild, tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdx1WithImage, tagIdx2WithImage},
},
{
name: "gallery_count_equals_1_depth_1",
buildFilter: models.TagFilterType{GalleryCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(1),
}},
includeIdxs: []int{tagIdxWithGallery, tagIdx3WithGallery, tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdx1WithGallery, tagIdx2WithGallery},
},
{
name: "gallery_count_equals_1_depth_-1",
buildFilter: models.TagFilterType{GalleryCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(-1),
}},
includeIdxs: []int{tagIdxWithGallery, tagIdx3WithGallery, tagIdxWithGrandChild, tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdx1WithGallery, tagIdx2WithGallery},
},
{
name: "performer_count_equals_1_depth_1",
buildFilter: models.TagFilterType{PerformerCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(1),
}},
includeIdxs: []int{tagIdxWithPerformer, tagIdx1WithPerformer, tagIdxWithGrandChild, tagIdxWithParentAndChild},
excludeIdxs: []int{tagIdx2WithPerformer},
},
{
name: "performer_count_equals_1_depth_-1",
buildFilter: models.TagFilterType{PerformerCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(-1),
}},
includeIdxs: []int{tagIdxWithPerformer, tagIdx1WithPerformer, tagIdxWithGrandChild, tagIdxWithParentAndChild},
excludeIdxs: []int{tagIdx2WithPerformer},
},
{
name: "marker_count_equals_1_depth_1",
buildFilter: models.TagFilterType{MarkerCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(1),
}},
includeIdxs: []int{tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdxWithPrimaryMarkers, tagIdxWithMarkers, tagIdx2WithMarkers},
},
{
name: "marker_count_equals_1_depth_-1",
buildFilter: models.TagFilterType{MarkerCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(-1),
}},
includeIdxs: []int{tagIdxWithGrandChild, tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdxWithPrimaryMarkers, tagIdxWithMarkers, tagIdx2WithMarkers},
},
{
name: "group_count_equals_1_depth_1",
buildFilter: models.TagFilterType{GroupCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(1),
}},
includeIdxs: []int{tagIdxWithGroup, tagIdx3WithGroup, tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdx1WithGroup, tagIdx2WithGroup},
},
{
name: "group_count_equals_1_depth_-1",
buildFilter: models.TagFilterType{GroupCount: &models.HierarchicalCountInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
Depth: ptr(-1),
}},
includeIdxs: []int{tagIdxWithGroup, tagIdx3WithGroup, tagIdxWithGrandChild, tagIdxWithParentAndChild, tagIdxWithGrandParent},
excludeIdxs: []int{tagIdx1WithGroup, tagIdx2WithGroup},
},
}
verifyTagSceneCount(t, countCriterion)
for _, tt := range tests {
runWithRollbackTxn(t, tt.name, func(t *testing.T, ctx context.Context) {
qb := db.Tag
tagFilter := &tt.buildFilter
countCriterion.Modifier = models.CriterionModifierNotEquals
verifyTagSceneCount(t, countCriterion)
tags, _, err := qb.Query(ctx, tagFilter, nil)
if err != nil {
t.Fatalf("%s: Error querying tag: %v", tt.name, err)
}
countCriterion.Modifier = models.CriterionModifierLessThan
verifyTagSceneCount(t, countCriterion)
ids := tagsToIDs(tags)
include := indexesToIDs(tagIDs, tt.includeIdxs)
exclude := indexesToIDs(tagIDs, tt.excludeIdxs)
countCriterion.Value = 0
countCriterion.Modifier = models.CriterionModifierGreaterThan
verifyTagSceneCount(t, countCriterion)
}
for _, id := range include {
assert.Contains(t, ids, id, "%s: expected id %d to be included", tt.name, id)
}
func verifyTagSceneCount(t *testing.T, sceneCountCriterion models.IntCriterionInput) {
withTxn(func(ctx context.Context) error {
qb := db.Tag
tagFilter := models.TagFilterType{
SceneCount: &sceneCountCriterion,
}
tags, _, err := qb.Query(ctx, &tagFilter, nil)
if err != nil {
t.Errorf("Error querying tag: %s", err.Error())
}
for _, tag := range tags {
verifyInt(t, getTagSceneCount(tag.ID), sceneCountCriterion)
}
return nil
})
}
func TestTagQueryMarkerCount(t *testing.T) {
countCriterion := models.IntCriterionInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
for _, id := range exclude {
assert.NotContains(t, ids, id, "%s: expected id %d to be excluded", tt.name, id)
}
})
}
verifyTagMarkerCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierNotEquals
verifyTagMarkerCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierLessThan
verifyTagMarkerCount(t, countCriterion)
countCriterion.Value = 0
countCriterion.Modifier = models.CriterionModifierGreaterThan
verifyTagMarkerCount(t, countCriterion)
}
func verifyTagMarkerCount(t *testing.T, markerCountCriterion models.IntCriterionInput) {
withTxn(func(ctx context.Context) error {
qb := db.Tag
tagFilter := models.TagFilterType{
MarkerCount: &markerCountCriterion,
}
tags, _, err := qb.Query(ctx, &tagFilter, nil)
if err != nil {
t.Errorf("Error querying tag: %s", err.Error())
}
for _, tag := range tags {
verifyInt(t, getTagMarkerCount(tag.ID), markerCountCriterion)
}
return nil
})
}
func TestTagQueryImageCount(t *testing.T) {
countCriterion := models.IntCriterionInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
}
verifyTagImageCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierNotEquals
verifyTagImageCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierLessThan
verifyTagImageCount(t, countCriterion)
countCriterion.Value = 0
countCriterion.Modifier = models.CriterionModifierGreaterThan
verifyTagImageCount(t, countCriterion)
}
func verifyTagImageCount(t *testing.T, imageCountCriterion models.IntCriterionInput) {
withTxn(func(ctx context.Context) error {
qb := db.Tag
tagFilter := models.TagFilterType{
ImageCount: &imageCountCriterion,
}
tags, _, err := qb.Query(ctx, &tagFilter, nil)
if err != nil {
t.Errorf("Error querying tag: %s", err.Error())
}
for _, tag := range tags {
verifyInt(t, getTagImageCount(tag.ID), imageCountCriterion)
}
return nil
})
}
func TestTagQueryGalleryCount(t *testing.T) {
countCriterion := models.IntCriterionInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
}
verifyTagGalleryCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierNotEquals
verifyTagGalleryCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierLessThan
verifyTagGalleryCount(t, countCriterion)
countCriterion.Value = 0
countCriterion.Modifier = models.CriterionModifierGreaterThan
verifyTagGalleryCount(t, countCriterion)
}
func verifyTagGalleryCount(t *testing.T, imageCountCriterion models.IntCriterionInput) {
withTxn(func(ctx context.Context) error {
qb := db.Tag
tagFilter := models.TagFilterType{
GalleryCount: &imageCountCriterion,
}
tags, _, err := qb.Query(ctx, &tagFilter, nil)
if err != nil {
t.Errorf("Error querying tag: %s", err.Error())
}
for _, tag := range tags {
verifyInt(t, getTagGalleryCount(tag.ID), imageCountCriterion)
}
return nil
})
}
func TestTagQueryPerformerCount(t *testing.T) {
countCriterion := models.IntCriterionInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
}
verifyTagPerformerCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierNotEquals
verifyTagPerformerCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierLessThan
verifyTagPerformerCount(t, countCriterion)
countCriterion.Value = 0
countCriterion.Modifier = models.CriterionModifierGreaterThan
verifyTagPerformerCount(t, countCriterion)
}
func verifyTagPerformerCount(t *testing.T, imageCountCriterion models.IntCriterionInput) {
withTxn(func(ctx context.Context) error {
qb := db.Tag
tagFilter := models.TagFilterType{
PerformerCount: &imageCountCriterion,
}
tags, _, err := qb.Query(ctx, &tagFilter, nil)
if err != nil {
t.Errorf("Error querying tag: %s", err.Error())
}
for _, tag := range tags {
verifyInt(t, getTagPerformerCount(tag.ID), imageCountCriterion)
}
return nil
})
}
func TestTagQueryStudioCount(t *testing.T) {
countCriterion := models.IntCriterionInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
}
verifyTagStudioCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierNotEquals
verifyTagStudioCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierLessThan
verifyTagStudioCount(t, countCriterion)
countCriterion.Value = 0
countCriterion.Modifier = models.CriterionModifierGreaterThan
verifyTagStudioCount(t, countCriterion)
}
func verifyTagStudioCount(t *testing.T, imageCountCriterion models.IntCriterionInput) {
withTxn(func(ctx context.Context) error {
qb := db.Tag
tagFilter := models.TagFilterType{
StudioCount: &imageCountCriterion,
}
tags, _, err := qb.Query(ctx, &tagFilter, nil)
if err != nil {
t.Errorf("Error querying tag: %s", err.Error())
}
for _, tag := range tags {
verifyInt(t, getTagStudioCount(tag.ID), imageCountCriterion)
}
return nil
})
}
func TestTagQueryParentCount(t *testing.T) {
countCriterion := models.IntCriterionInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
}
verifyTagParentCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierNotEquals
verifyTagParentCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierLessThan
verifyTagParentCount(t, countCriterion)
countCriterion.Value = 0
countCriterion.Modifier = models.CriterionModifierGreaterThan
verifyTagParentCount(t, countCriterion)
}
func verifyTagParentCount(t *testing.T, sceneCountCriterion models.IntCriterionInput) {
withTxn(func(ctx context.Context) error {
qb := db.Tag
tagFilter := models.TagFilterType{
ParentCount: &sceneCountCriterion,
}
tags := queryTags(ctx, t, qb, &tagFilter, nil)
if len(tags) == 0 {
t.Error("Expected at least one tag")
}
for _, tag := range tags {
verifyInt(t, getTagParentCount(tag.ID), sceneCountCriterion)
}
return nil
})
}
func TestTagQueryChildCount(t *testing.T) {
countCriterion := models.IntCriterionInput{
Value: 1,
Modifier: models.CriterionModifierEquals,
}
verifyTagChildCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierNotEquals
verifyTagChildCount(t, countCriterion)
countCriterion.Modifier = models.CriterionModifierLessThan
verifyTagChildCount(t, countCriterion)
countCriterion.Value = 0
countCriterion.Modifier = models.CriterionModifierGreaterThan
verifyTagChildCount(t, countCriterion)
}
func verifyTagChildCount(t *testing.T, sceneCountCriterion models.IntCriterionInput) {
withTxn(func(ctx context.Context) error {
qb := db.Tag
tagFilter := models.TagFilterType{
ChildCount: &sceneCountCriterion,
}
tags := queryTags(ctx, t, qb, &tagFilter, nil)
if len(tags) == 0 {
t.Error("Expected at least one tag")
}
for _, tag := range tags {
verifyInt(t, getTagChildCount(tag.ID), sceneCountCriterion)
}
return nil
})
}
func TestTagQueryParent(t *testing.T) {

View File

@@ -54,9 +54,6 @@ const useContainingGroupFilterHook = (
filter.criteria.push(groupCriterion);
}
filter.sortBy = "sub_group_order";
filter.sortDirection = GQL.SortDirectionEnum.Asc;
return filter;
};
};
@@ -67,18 +64,6 @@ interface IGroupSubGroupsPanel {
extraOperations?: IItemListOperation<GQL.FindGroupsQueryResult>[];
}
const defaultFilter = (() => {
const sortBy = "sub_group_order";
const ret = new ListFilterModel(GQL.FilterMode.Groups, undefined, {
defaultSortBy: sortBy,
});
// unset the sort by so that its not included in the URL
ret.sortBy = undefined;
return ret;
})();
export const GroupSubGroupsPanel: React.FC<IGroupSubGroupsPanel> =
PatchComponent(
"GroupSubGroupsPanel",
@@ -163,7 +148,8 @@ export const GroupSubGroupsPanel: React.FC<IGroupSubGroupsPanel> =
<>
{modal}
<FilteredGroupList
defaultFilter={defaultFilter}
defaultSort="sub_group_order"
manualSortBy="sub_group_order"
filterHook={filterHook}
alterQuery={active}
fromGroupId={group.id}

View File

@@ -147,13 +147,15 @@ const SidebarContent: React.FC<{
interface IGroupListContext {
filterHook?: (filter: ListFilterModel) => ListFilterModel;
defaultFilter?: ListFilterModel;
view?: View;
alterQuery?: boolean;
}
interface IGroupList extends IGroupListContext {
defaultSort?: string;
fromGroupId?: string;
// specifies the sort by value that allows reordering
manualSortBy?: string;
onMove?: (srcIds: string[], targetId: string, after: boolean) => void;
otherOperations?: IItemListOperation<GQL.FindGroupsQueryResult>[];
}
@@ -204,19 +206,16 @@ export const FilteredGroupList = PatchComponent(
const searchFocus = useFocus();
const {
defaultSort,
filterHook,
view,
alterQuery,
manualSortBy,
onMove,
fromGroupId,
otherOperations: providedOperations = [],
defaultFilter,
} = props;
const withSidebar = view !== View.GroupSubGroups;
const filterable = view !== View.GroupSubGroups;
const sortable = view !== View.GroupSubGroups;
// States
const {
showSidebar,
@@ -230,7 +229,7 @@ export const FilteredGroupList = PatchComponent(
useFilteredItemList({
filterStateProps: {
filterMode: GQL.FilterMode.Groups,
defaultFilter,
defaultSort,
view,
useURL: alterQuery,
},
@@ -379,6 +378,8 @@ export const FilteredGroupList = PatchComponent(
// render
if (sidebarStateLoading) return null;
const canMove = manualSortBy && onMove && filter.sortBy === manualSortBy;
const operations = (
<ListOperations
items={items.length}
@@ -402,8 +403,6 @@ export const FilteredGroupList = PatchComponent(
operationComponent={operations}
view={view}
zoomable
filterable={filterable}
sortable={sortable}
/>
<FilterTags
@@ -435,7 +434,7 @@ export const FilteredGroupList = PatchComponent(
selectedIds={selectedIds}
onSelectChange={onSelectChange}
fromGroupId={fromGroupId}
onMove={onMove}
onMove={canMove ? onMove : undefined}
/>
</LoadedContent>
@@ -455,10 +454,6 @@ export const FilteredGroupList = PatchComponent(
</>
);
if (!withSidebar) {
return content;
}
return (
<div
className={cx("item-list-container group-list", {

View File

@@ -99,6 +99,10 @@ export const FilteredListToolbar: React.FC<IFilteredListToolbar> = ({
sortable = true,
}) => {
const filterOptions = filter.options;
// Something in the popper layout for groups.sub-groups tab to double calculates the offset
// causing the dropdown to be misaligned. Portal to document.body to fix this.
const menuPortalTarget =
typeof document !== "undefined" ? document.body : undefined;
const { setDisplayMode, setZoom } = useFilterOperations({
filter,
setFilter,
@@ -142,6 +146,7 @@ export const FilteredListToolbar: React.FC<IFilteredListToolbar> = ({
filter={filter}
onSetFilter={setFilter}
view={view}
menuPortalTarget={menuPortalTarget}
/>
<FilterButton
onClick={() => showEditFilter()}

View File

@@ -20,9 +20,9 @@ import {
FilterMode,
GalleryFilterType,
GroupFilterType,
HierarchicalCountInput,
ImageFilterType,
InputMaybe,
IntCriterionInput,
PerformerFilterType,
SceneFilterType,
SceneMarkerFilterType,
@@ -520,18 +520,18 @@ export function makeQueryVariables(query: string, extraProps: object) {
interface IFilterType {
scenes_filter?: InputMaybe<SceneFilterType>;
scene_count?: InputMaybe<IntCriterionInput>;
scene_count?: InputMaybe<HierarchicalCountInput>;
performers_filter?: InputMaybe<PerformerFilterType>;
performer_count?: InputMaybe<IntCriterionInput>;
performer_count?: InputMaybe<HierarchicalCountInput>;
galleries_filter?: InputMaybe<GalleryFilterType>;
gallery_count?: InputMaybe<IntCriterionInput>;
gallery_count?: InputMaybe<HierarchicalCountInput>;
images_filter?: InputMaybe<ImageFilterType>;
image_count?: InputMaybe<IntCriterionInput>;
image_count?: InputMaybe<HierarchicalCountInput>;
groups_filter?: InputMaybe<GroupFilterType>;
group_count?: InputMaybe<IntCriterionInput>;
group_count?: InputMaybe<HierarchicalCountInput>;
studios_filter?: InputMaybe<StudioFilterType>;
studio_count?: InputMaybe<IntCriterionInput>;
marker_count?: InputMaybe<IntCriterionInput>;
studio_count?: InputMaybe<HierarchicalCountInput>;
marker_count?: InputMaybe<HierarchicalCountInput>;
markers_filter?: InputMaybe<SceneMarkerFilterType>;
}
@@ -554,6 +554,7 @@ export function setObjectFilter(
out.scene_count = {
modifier: CriterionModifier.GreaterThan,
value: 0,
depth: -1,
};
break;
}
@@ -565,6 +566,7 @@ export function setObjectFilter(
out.performer_count = {
modifier: CriterionModifier.GreaterThan,
value: 0,
depth: -1,
};
break;
}
@@ -576,6 +578,7 @@ export function setObjectFilter(
out.gallery_count = {
modifier: CriterionModifier.GreaterThan,
value: 0,
depth: -1,
};
break;
}
@@ -587,6 +590,7 @@ export function setObjectFilter(
out.image_count = {
modifier: CriterionModifier.GreaterThan,
value: 0,
depth: -1,
};
break;
}
@@ -598,6 +602,7 @@ export function setObjectFilter(
out.group_count = {
modifier: CriterionModifier.GreaterThan,
value: 0,
depth: -1,
};
break;
}
@@ -609,6 +614,7 @@ export function setObjectFilter(
out.studio_count = {
modifier: CriterionModifier.GreaterThan,
value: 0,
depth: -1,
};
break;
}
@@ -620,6 +626,7 @@ export function setObjectFilter(
out.marker_count = {
modifier: CriterionModifier.GreaterThan,
value: 0,
depth: -1,
};
break;
}

View File

@@ -273,6 +273,7 @@ export const ScrapedPerformersRow: React.FC<
}}
values={selectValue}
ageFromDate={ageFromDate}
hoverPlacementLabel="bottom"
/>
);
}

View File

@@ -57,12 +57,14 @@ export const StudioEditPanel: React.FC<IStudioEditPanel> = ({
const [isLoading, setIsLoading] = useState(false);
const [parentStudio, setParentStudio] = useState<Studio | null>(null);
const [childStudios, setChildStudios] = useState<Studio[]>([]);
const schema = yup.object({
name: yup.string().required(),
urls: yup.array(yup.string().required()).defined(),
details: yup.string().ensure(),
parent_id: yup.string().required().nullable(),
child_ids: yup.array(yup.string().required()).defined(),
aliases: yupRequiredStringArray(intl).defined(),
tag_ids: yup.array(yup.string().required()).defined(),
ignore_auto_tag: yup.boolean().defined(),
@@ -77,6 +79,7 @@ export const StudioEditPanel: React.FC<IStudioEditPanel> = ({
urls: studio.urls ?? [],
details: studio.details ?? "",
parent_id: studio.parent_studio?.id ?? null,
child_ids: (studio.child_studios ?? []).map((child) => child.id),
aliases: studio.aliases ?? [],
tag_ids: (studio.tags ?? []).map((t) => t.id),
ignore_auto_tag: studio.ignore_auto_tag ?? false,
@@ -112,6 +115,14 @@ export const StudioEditPanel: React.FC<IStudioEditPanel> = ({
formik.setFieldValue("parent_id", item ? item.id : null);
}
function onSetChildStudios(items: Studio[]) {
setChildStudios(items);
formik.setFieldValue(
"child_ids",
items.map((item) => item.id)
);
}
const encodingImage = ImageUtils.usePasteImage((imageData) =>
formik.setFieldValue("image", imageData)
);
@@ -128,6 +139,17 @@ export const StudioEditPanel: React.FC<IStudioEditPanel> = ({
);
}, [studio.parent_studio]);
useEffect(() => {
setChildStudios(
(studio.child_studios ?? []).map((childStudio) => ({
id: childStudio.id,
name: childStudio.name,
aliases: [],
image_path: childStudio.image_path,
}))
);
}, [studio.child_studios]);
useEffect(() => {
setImage(formik.values.image);
}, [formik.values.image, setImage]);
@@ -199,12 +221,35 @@ export const StudioEditPanel: React.FC<IStudioEditPanel> = ({
onSetParentStudio(items.length > 0 ? items[0] : null)
}
values={parentStudio ? [parentStudio] : []}
excludeIds={[
...(studio?.id ? [studio.id] : []),
...formik.values.child_ids,
]}
/>
);
return renderField("parent_id", title, control);
}
function renderSubStudiosField() {
const title = intl.formatMessage({ id: "subsidiary_studios" });
const control = (
<StudioSelect
isMulti
onSelect={onSetChildStudios}
values={childStudios.filter((childStudio) =>
formik.values.child_ids.includes(childStudio.id)
)}
excludeIds={[
...(studio?.id ? [studio.id] : []),
...(formik.values.parent_id ? [formik.values.parent_id] : []),
]}
/>
);
return renderField("child_ids", title, control);
}
function renderTagsField() {
const title = intl.formatMessage({ id: "tags" });
return renderField("tag_ids", title, tagsControl());
@@ -246,6 +291,7 @@ export const StudioEditPanel: React.FC<IStudioEditPanel> = ({
{renderStringListField("urls")}
{renderInputField("details", "textarea")}
{renderParentStudioField()}
{renderSubStudiosField()}
{renderTagsField()}
{renderStashIDsField(
"stash_ids",

View File

@@ -86,5 +86,7 @@ export const PERFORMER_FIELDS = [
"details",
];
export const PERFORMER_MERGEABLE_FIELDS = ["aliases", "urls"];
export const STUDIO_FIELDS = ["name", "image", "url", "parent_studio"];
export const TAG_FIELDS = ["name", "description", "aliases", "parent_tags"];

View File

@@ -1,5 +1,14 @@
import React, { useEffect, useMemo, useRef, useState } from "react";
import { Button, Card, Form, InputGroup, ProgressBar } from "react-bootstrap";
import {
Button,
Card,
Col,
Collapse,
Form,
InputGroup,
ProgressBar,
Row,
} from "react-bootstrap";
import { FormattedMessage, useIntl } from "react-intl";
import { Link } from "react-router-dom";
import { HashLink } from "react-router-hash-link";
@@ -19,10 +28,21 @@ import { useConfigurationContext } from "src/hooks/Config";
import StashSearchResult from "./StashSearchResult";
import TaggerConfig, { ConfigButton } from "../TaggerConfig";
import { ITaggerConfig, PERFORMER_FIELDS } from "../constants";
import {
ITaggerConfig,
PERFORMER_FIELDS,
PERFORMER_MERGEABLE_FIELDS,
} from "../constants";
import PerformerModal from "../PerformerModal";
import { useUpdatePerformer } from "../queries";
import { faStar, faTags } from "@fortawesome/free-solid-svg-icons";
import {
faCheck,
faPlus,
faStar,
faTags,
faTimes,
} from "@fortawesome/free-solid-svg-icons";
import { Icon } from "src/components/Shared/Icon";
import { mergeStashIDs } from "src/utils/stashbox";
import { separateNamesAndStashIds } from "src/utils/stashIds";
import { ExternalLink } from "src/components/Shared/ExternalLink";
@@ -36,11 +56,19 @@ type JobFragment = Pick<
const CLASSNAME = "PerformerTagger";
type FieldMode = "overwrite" | "merge" | "skip";
interface IPerformerBatchUpdateModal {
performers: GQL.PerformerDataFragment[];
isIdle: boolean;
selectedEndpoint: { endpoint: string; index: number };
onBatchUpdate: (queryAll: boolean, refresh: boolean) => void;
excludedFields: string[];
onBatchUpdate: (
queryAll: boolean,
refresh: boolean,
excludedFields: string[],
mergeFields: string[]
) => void;
close: () => void;
}
@@ -48,14 +76,36 @@ const PerformerBatchUpdateModal: React.FC<IPerformerBatchUpdateModal> = ({
performers,
isIdle,
selectedEndpoint,
excludedFields: initialExcludedFields,
onBatchUpdate,
close,
}) => {
const intl = useIntl();
const [queryAll, setQueryAll] = useState(false);
const [refresh, setRefresh] = useState(false);
const [showFieldSelect, setShowFieldSelect] = useState(false);
const [fieldModes, setFieldModes] = useState<Record<string, FieldMode>>(() =>
PERFORMER_FIELDS.reduce(
(acc, field) => ({
...acc,
[field]: initialExcludedFields.includes(field) ? "skip" : "overwrite",
}),
{} as Record<string, FieldMode>
)
);
const excludedFieldsList = useMemo(
() => PERFORMER_FIELDS.filter((f) => fieldModes[f] === "skip"),
[fieldModes]
);
const mergeFieldsList = useMemo(
() => PERFORMER_FIELDS.filter((f) => fieldModes[f] === "merge"),
[fieldModes]
);
const { data: allPerformers } = GQL.useFindPerformersQuery({
variables: {
performer_filter: {
@@ -87,6 +137,52 @@ const PerformerBatchUpdateModal: React.FC<IPerformerBatchUpdateModal> = ({
).length;
}, [queryAll, refresh, performers, allPerformers, selectedEndpoint.endpoint]);
const cycleFieldMode = (field: string) => {
const isMergeable = PERFORMER_MERGEABLE_FIELDS.includes(field);
const current = fieldModes[field] ?? "overwrite";
let next: FieldMode;
if (isMergeable) {
const cycle: FieldMode[] = ["overwrite", "merge", "skip"];
next = cycle[(cycle.indexOf(current) + 1) % cycle.length];
} else {
next = current === "overwrite" ? "skip" : "overwrite";
}
setFieldModes({ ...fieldModes, [field]: next });
};
const getFieldIcon = (mode: FieldMode) => {
switch (mode) {
case "overwrite":
return faCheck;
case "merge":
return faPlus;
case "skip":
return faTimes;
}
};
const getFieldClass = (mode: FieldMode) => {
switch (mode) {
case "overwrite":
return "text-success";
case "merge":
return "text-info";
case "skip":
return "text-muted";
}
};
const getFieldLabel = (mode: FieldMode) => {
switch (mode) {
case "overwrite":
return intl.formatMessage({ id: "actions.overwrite" });
case "merge":
return intl.formatMessage({ id: "actions.merge" });
case "skip":
return intl.formatMessage({ id: "actions.skip" });
}
};
return (
<ModalComponent
show
@@ -98,7 +194,8 @@ const PerformerBatchUpdateModal: React.FC<IPerformerBatchUpdateModal> = ({
text: intl.formatMessage({
id: "performer_tagger.update_performers",
}),
onClick: () => onBatchUpdate(queryAll, refresh),
onClick: () =>
onBatchUpdate(queryAll, refresh, excludedFieldsList, mergeFieldsList),
}}
cancel={{
text: intl.formatMessage({ id: "actions.cancel" }),
@@ -165,6 +262,58 @@ const PerformerBatchUpdateModal: React.FC<IPerformerBatchUpdateModal> = ({
<FormattedMessage id="performer_tagger.refreshing_will_update_the_data" />
</Form.Text>
</Form.Group>
<Form.Group>
<Form.Label>
<h6>
<FormattedMessage id="performer_tagger.field_options" />
</h6>
</Form.Label>
<Form.Text className="mb-2 d-block">
<FormattedMessage id="performer_tagger.field_options_description" />
</Form.Text>
<Button
onClick={() => setShowFieldSelect(!showFieldSelect)}
className="mt-1"
size="sm"
>
<FormattedMessage id="performer_tagger.configure_fields" />
</Button>
<Collapse in={showFieldSelect}>
<div className="mt-2">
<Row>
{PERFORMER_FIELDS.map((field) => {
const mode = fieldModes[field] ?? "overwrite";
return (
<Col xs={6} className="mb-1" key={field}>
<Button
onClick={() => cycleFieldMode(field)}
variant="secondary"
size="sm"
className={getFieldClass(mode)}
title={getFieldLabel(mode)}
>
<Icon icon={getFieldIcon(mode)} />
</Button>
<span className="ml-2">
<FormattedMessage id={field} />
</span>
</Col>
);
})}
</Row>
<div className="mt-2 small text-muted">
<Icon icon={faCheck} className="text-success" />{" "}
<FormattedMessage id="actions.overwrite" />
{" | "}
<Icon icon={faPlus} className="text-info" />{" "}
<FormattedMessage id="actions.merge" />
{" | "}
<Icon icon={faTimes} className="text-muted" />{" "}
<FormattedMessage id="actions.skip" />
</div>
</div>
</Collapse>
</Form.Group>
<b>
<FormattedMessage
id="performer_tagger.number_of_performers_will_be_processed"
@@ -240,7 +389,12 @@ interface IPerformerTaggerListProps {
isIdle: boolean;
config: ITaggerConfig;
onBatchAdd: (performerInput: string) => void;
onBatchUpdate: (ids: string[] | undefined, refresh: boolean) => void;
onBatchUpdate: (
ids: string[] | undefined,
refresh: boolean,
excludedFields: string[],
mergeFields: string[]
) => void;
}
const PerformerTaggerList: React.FC<IPerformerTaggerListProps> = ({
@@ -333,8 +487,18 @@ const PerformerTaggerList: React.FC<IPerformerTaggerListProps> = ({
setShowBatchAdd(false);
}
const handleBatchUpdate = (queryAll: boolean, refresh: boolean) => {
onBatchUpdate(!queryAll ? performers.map((p) => p.id) : undefined, refresh);
const handleBatchUpdate = (
queryAll: boolean,
refresh: boolean,
excludedFields: string[],
mergeFields: string[]
) => {
onBatchUpdate(
!queryAll ? performers.map((p) => p.id) : undefined,
refresh,
excludedFields,
mergeFields
);
setShowBatchUpdate(false);
};
@@ -601,6 +765,7 @@ const PerformerTaggerList: React.FC<IPerformerTaggerListProps> = ({
isIdle={isIdle}
selectedEndpoint={selectedEndpoint}
performers={performers}
excludedFields={config.excludedPerformerFields ?? []}
onBatchUpdate={handleBatchUpdate}
/>
)}
@@ -706,13 +871,19 @@ export const PerformerTagger: React.FC<ITaggerProps> = ({ performers }) => {
}
}
async function batchUpdate(ids: string[] | undefined, refresh: boolean) {
async function batchUpdate(
ids: string[] | undefined,
refresh: boolean,
excludedFields: string[],
mergeFields: string[]
) {
if (config && selectedEndpoint) {
const ret = await mutateStashBoxBatchPerformerTag({
ids: ids,
endpoint: selectedEndpointIndex,
refresh,
exclude_fields: config.excludedPerformerFields ?? [],
exclude_fields: excludedFields,
merge_fields: mergeFields.length > 0 ? mergeFields : undefined,
createParent: false,
});

View File

@@ -13,6 +13,8 @@ For best results, images in zip file should be stored without compression (copy,
> **⚠️ Note:** AVIF files in ZIP archives are currently unsupported.
> ** Note:** JPEG XL (`.jxl`) is supported, including inside ZIP archives, and requires an FFmpeg built with `libjxl` (or [libvips](https://www.libvips.org/)). Perceptual hashes are not generated for JXL inside ZIP archives.
If a filename of an image in the gallery zip file ends with `cover.jpg`, it will be treated like a cover and presented first in the gallery view page and as a gallery cover in the gallery list view. If more than one images match the name the first one found in natural sort order is selected.
You can also manually select any image from a gallery as its cover. On the gallery details page, select the desired cover image, and then select **Set as Cover** in the ⋯ menu.

View File

@@ -1378,8 +1378,11 @@
"any_names_entered_will_be_queried": "Any names entered will be queried from the remote Stash-Box instance and added if found. Only exact matches will be considered a match.",
"batch_add_performers": "Batch Add Performers",
"batch_update_performers": "Batch Update Performers",
"configure_fields": "Configure Fields",
"current_page": "Current page",
"failed_to_save_performer": "Failed to save performer \"{performer}\"",
"field_options": "Field Options",
"field_options_description": "Choose how each field is updated: overwrite replaces existing data, merge adds to it, skip leaves it unchanged.",
"name_already_exists": "Name already exists",
"network_error": "Network Error",
"no_results_found": "No results found.",
@@ -1617,6 +1620,7 @@
"sub_group_count": "Sub-Group Count",
"sub_group_of": "Sub-group of {parent}",
"sub_group_order": "Sub-Group Order",
"sub_group_description": "Sub-Group Description",
"sub_groups": "Sub-Groups",
"sub_tag_count": "Sub-Tag Count",
"sub_tag_of": "Sub-tag of {parent}",

View File

@@ -29,6 +29,10 @@ const defaultSortBy = "path";
const sortByOptions = ["date", ...MediaSortByOptions]
.map(ListFilterOptions.createSortBy)
.concat([
{
messageID: "performer_age",
value: "performer_age",
},
{
messageID: "image_count",
value: "images_count",

View File

@@ -28,6 +28,7 @@ const sortByOptions = [
"duration",
"rating",
"tag_count",
"sub_group_description",
"sub_group_order",
]
.map(ListFilterOptions.createSortBy)