This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-5568-40526b8e1783e7782ce6043e7ca72840b7fbe04d in repository https://gitbox.apache.org/repos/asf/texera.git
commit 4eb0068cb0fca121aae4cdba7a70177b32ad8934 Author: Elliot Lin <[email protected]> AuthorDate: Tue Jul 7 12:01:49 2026 -0700 feat(frontend): add task-aware field visibility and preview to HuggingFace property editor (#5568) ### What changes were proposed in this PR? Wire up the HuggingFace operator's property editor so that selecting a task dynamically controls which fields are visible and shows a media preview card. This is the PR that makes the formly components from PRs #5566/ #5567 user-visible by mapping operator fields to custom field types in `jsonSchemaMapIntercept`. Changes to `operator-property-edit-frame.component.ts`: - Map `modelId` → `huggingface` formly type, `imageInput` → `huggingface-image-upload`, `audioInput` → `huggingface-audio-upload` - Hide the `task` field (it is controlled by the HuggingFaceComponent's task dropdown instead) - ~13 field visibility rules via formly expressions that show/hide fields based on the selected task (e.g., `imageInput` only for image tasks, `contextColumn` only for question-answering, `systemPrompt`/`maxNewTokens`/`temperature` only for text-generation) - 3 custom validators: `requiredImageInput`, `requiredAudioInput`, `requiredPromptColumn` — each checks whether the direct input or the corresponding column selector is filled - Removed `validation: { show: true }` from imageInput, audioInput, and promptColumn so errors only appear after the field is touched, not on a freshly-added operator - Added `image-to-video` to `imageInputTasks` so the image input field appears for that task - Task preview cards with sample media for 22 task types across 4 kinds (image, video, audio, text), plus a fallback for unknown tasks (sample-video.mp4 and sample-audio.wav are self-created) - Operator description truncation — clamps to 3 lines with CSS and adds a tooltip for the full text on hover Changes to test infrastructure: - `mockHuggingFaceSchema` in `mock-operator-metadata.data.ts` (added to `mockOperatorSchemaList`) - `mockHuggingFacePredicate` in `mock-workflow-data.ts` ### Any related issues, documentation, discussions? - Tracking issue: #5316 - Closes: #5316 - Stacked on: #5567 - Parent issue: #5041 ### How was this PR tested? **`operator-property-edit-frame.component.spec.ts`** — 123 new tests - **Task preview**: all 20+ tasks (text-generation, image-classification, text-to-speech, text-to-video, ASR, image-to-image, object-detection, image-segmentation, image-to-text, document-question-answering, zero-shot-image-classification, image-text-to-text, sentence-similarity, text-ranking, translation, summarization, text-classification, token-classification, table-question-answering, feature-extraction, fill-mask), unknown task fallback, empty/whitespace task, non-HF operator, deleted operator guard, preview content validation (assetSrc, pills, outputBody) - **Field type assignments**: `modelId` → `huggingface`, `imageInput` → `huggingface-image-upload`, `audioInput` → `huggingface-audio-upload` - **Field visibility**: imageInput, audioInput, inputImageColumn, inputAudioColumn, promptColumn, contextColumn, candidateLabels, sentencesColumn, systemPrompt, maxNewTokens, temperature — tested across 15+ tasks including mixed-input tasks (visual-question-answering, document-question-answering, image-text-to-text) and media-generation tasks (text-to-image, text-to-speech) - **Undefined task visibility**: all 11 fields hide when no task is selected - **Validators**: requiredPromptColumn, requiredImageInput, requiredAudioInput — pass/fail for empty, whitespace, model values, formControl values, inputColumn alternatives, zero-shot-image-classification edge cases, error message strings - **getSelectedTask fallback**: model.task → form.get → formControl.parent chain - **Operator description**: tooltip rendering, hidden when unset - **Task field hidden for HF operators** **`hugging-face-image-upload.component.spec.ts`** — 27 new tests - **Successful upload**: happy path (fileName, formControl, model, dirty/touched), clears prior error - **Compression pipeline**: FileReader error, ArrayBuffer result, null result, Image onerror, null canvas context, quality reduction, dimension reduction, never fits, non-image toDataURL, 1x1 pixel, extreme aspect ratio - **clearImage edge cases**: non-string key guard, dirty/touched - **Consecutive uploads**: second upload replaces first **`hugging-face.component.spec.ts`** — 11 new tests - **Pagination edge cases**: prevPage at page 0, goToPage on empty list - **Task state snapshots**: defaults on first visit to new task, preserve across switches - **Server search edge cases**: error handling, query superseding - **getCurrentTaskTag / model selection / retryLoad** edge cases **`hugging-face-audio-upload.component.spec.ts`** — 7 new tests - **formControl state**: dirty/touched after upload and clear - **Model update**: non-string key skips model write - **previewSrc edge cases**: non-audio data URL, alternate audio MIME type - **Consecutive uploads**: replacement - **revokePreviewUrl**: no throw without preview Run with `ng test`. ### Was this PR authored or co-authored using generative AI tooling? Co-authored with Claude Opus 4.6 --------- Co-authored-by: Elliot <[email protected]> Co-authored-by: Anish Shivamurthy <[email protected]> Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> --- .../huggingFace/codegen/ImageTaskCodegen.scala | 24 +- .../hugging-face-audio-upload.component.spec.ts | 93 ++ .../hugging-face-image-upload.component.spec.ts | 460 ++++++++ .../hugging-face/hugging-face.component.spec.ts | 218 ++++ .../operator-property-edit-frame.component.html | 82 ++ .../operator-property-edit-frame.component.scss | 127 +++ .../operator-property-edit-frame.component.spec.ts | 1188 +++++++++++++++++++- .../operator-property-edit-frame.component.ts | 472 +++++++- .../mock-operator-metadata.data.ts | 31 + .../workflow-graph/model/mock-workflow-data.ts | 11 + frontend/src/assets/sample-audio.wav | Bin 0 -> 170444 bytes frontend/src/assets/sample-video.mp4 | Bin 0 -> 117816 bytes 12 files changed, 2687 insertions(+), 19 deletions(-) diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegen.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegen.scala index c5c4a2669c..5a5ee0a937 100644 --- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegen.scala +++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegen.scala @@ -90,26 +90,16 @@ object ImageTaskCodegen extends TaskCodegen { | use_raw_binary_body = True | raw_binary_headers = image_headers | elif task == "zero-shot-image-classification": - | # Zero-shot requires the caller to supply candidate labels. - | # We reuse the prompt column as a comma-separated label list so - | # the task is shippable without a dedicated operator field. - | # TODO: replace with a first-class `candidateLabels` field once - | # the property panel supports task-specific inputs. - | # - | # Fail fast if usable labels can't be derived. Both modes lead to - | # a meaningless inference call: - | # 1. Empty prompt column -> labels = [] - | # The HF API rejects candidate_labels: [] with an opaque 400. - | # 2. Missing prompt column -> upstream sets prompt_value - | # to the fallback "What is shown in this image?", which has - | # no comma, so labels collapses to a single nonsense entry. - | # Zero-shot classification needs >= 2 candidate labels to be - | # meaningful — surface a configuration error in both cases. - | labels = [s.strip() for s in prompt_value.split(",") if s.strip()] + | # Prefer the dedicated candidateLabels property; fall back to + | # the prompt column for backward compatibility. + | label_source = (self.CANDIDATE_LABELS or "").strip() if self.CANDIDATE_LABELS else "" + | if not label_source and prompt_value: + | label_source = prompt_value + | labels = [s.strip() for s in label_source.split(",") if s.strip()] | if len(labels) < 2: | raise ValueError( | "zero-shot-image-classification requires at least 2 candidate " - | "labels: provide a comma-separated list in the prompt column." + | "labels: provide a comma-separated list in the Candidate Labels field." | ) | payload = { | "inputs": self._image_input_as_base64(current_image_bytes), diff --git a/frontend/src/app/workspace/component/hugging-face-audio-upload/hugging-face-audio-upload.component.spec.ts b/frontend/src/app/workspace/component/hugging-face-audio-upload/hugging-face-audio-upload.component.spec.ts index bb7ebeb619..f92300731a 100644 --- a/frontend/src/app/workspace/component/hugging-face-audio-upload/hugging-face-audio-upload.component.spec.ts +++ b/frontend/src/app/workspace/component/hugging-face-audio-upload/hugging-face-audio-upload.component.spec.ts @@ -481,4 +481,97 @@ describe("HuggingFaceAudioUploadComponent", () => { httpTestingController.expectOne(r => r.url.includes("/huggingface/audio-preview")); }); }); + + // ── Upload marks formControl as dirty and touched ── + + describe("formControl state after upload", () => { + it("should mark formControl as dirty and touched after successful upload", async () => { + const file = new File(["audio-data"], "clip.wav", { type: "audio/wav" }); + const uploadPromise = component.onFileSelected(makeFileEvent(file)); + + const req = httpTestingController.expectOne(r => r.url.includes("/huggingface/upload-audio")); + req.flush({ path: "/tmp/clip.wav", fileName: "clip.wav" }); + await uploadPromise; + + expect(formControl.dirty).toBe(true); + expect(formControl.touched).toBe(true); + }); + + it("should mark formControl as dirty and touched after clear", () => { + formControl.setValue("/tmp/clip.wav"); + const input = document.createElement("input"); + component.clearAudio(input); + + expect(formControl.dirty).toBe(true); + expect(formControl.touched).toBe(true); + }); + }); + + // ── Upload updates model ── + + describe("model update on upload", () => { + it("should not update model when key is not a string", async () => { + const model: Record<string, unknown> = {}; + component.field = { formControl, key: 42 as any, model } as unknown as FieldTypeConfig; + + const file = new File(["audio-data"], "clip.wav", { type: "audio/wav" }); + const uploadPromise = component.onFileSelected(makeFileEvent(file)); + + const req = httpTestingController.expectOne(r => r.url.includes("/huggingface/upload-audio")); + req.flush({ path: "/tmp/clip.wav", fileName: "clip.wav" }); + await uploadPromise; + + expect(formControl.value).toBe("/tmp/clip.wav"); + expect(model[42 as any]).toBeUndefined(); + }); + }); + + // ── revokePreviewUrl no-op ── + + describe("revokePreviewUrl", () => { + it("should not throw when destroying without any preview", () => { + expect(() => component.ngOnDestroy()).not.toThrow(); + }); + }); + + // ── previewSrc with non-audio data URL ── + + describe("previewSrc edge cases", () => { + it("should return empty for non-audio data URL", () => { + formControl.setValue("data:image/png;base64,abc123"); + expect(component.previewSrc).toBe(""); + }); + + it("should return data:audio value with different audio type", () => { + const mp3DataUrl = "data:audio/mp3;base64,abc123"; + formControl.setValue(mp3DataUrl); + expect(component.previewSrc).toBe(mp3DataUrl); + }); + }); + + // ── Multiple consecutive uploads ── + + describe("consecutive uploads", () => { + it("should replace previous upload value with new upload", async () => { + // First upload + const file1 = new File(["audio-1"], "first.wav", { type: "audio/wav" }); + const upload1 = component.onFileSelected(makeFileEvent(file1)); + const req1 = httpTestingController.expectOne(r => r.url.includes("/huggingface/upload-audio")); + req1.flush({ path: "/tmp/first.wav", fileName: "first.wav" }); + await upload1; + + expect(formControl.value).toBe("/tmp/first.wav"); + expect(component.fileName).toBe("first.wav"); + + // Second upload + const file2 = new File(["audio-2"], "second.wav", { type: "audio/wav" }); + const upload2 = component.onFileSelected(makeFileEvent(file2)); + const req2 = httpTestingController.expectOne(r => r.url.includes("/huggingface/upload-audio")); + req2.flush({ path: "/tmp/second.wav", fileName: "second.wav" }); + await upload2; + + expect(formControl.value).toBe("/tmp/second.wav"); + expect(component.fileName).toBe("second.wav"); + }); + }); }); diff --git a/frontend/src/app/workspace/component/hugging-face-image-upload/hugging-face-image-upload.component.spec.ts b/frontend/src/app/workspace/component/hugging-face-image-upload/hugging-face-image-upload.component.spec.ts index 6bd947ef0e..e22a4f5715 100644 --- a/frontend/src/app/workspace/component/hugging-face-image-upload/hugging-face-image-upload.component.spec.ts +++ b/frontend/src/app/workspace/component/hugging-face-image-upload/hugging-face-image-upload.component.spec.ts @@ -143,4 +143,464 @@ describe("HuggingFaceImageUploadComponent", () => { expect((component.model as any).image).toBe(""); }); }); + + describe("hasImage edge cases", () => { + it("returns false for a server path string", () => { + component.formControl.setValue("/uploads/photo.png"); + expect(component.hasImage).toBe(false); + }); + + it("returns false for null value", () => { + component.formControl.setValue(null); + expect(component.hasImage).toBe(false); + }); + + it("returns false for non-image data URL", () => { + component.formControl.setValue("data:audio/wav;base64,AAA"); + expect(component.hasImage).toBe(false); + }); + + it("returns true for data:image/png URL", () => { + component.formControl.setValue("data:image/png;base64,AAA"); + expect(component.hasImage).toBe(true); + }); + }); + + describe("previewSrc", () => { + it("returns data URL when formControl has image", () => { + component.formControl.setValue("data:image/jpeg;base64,AAA"); + expect(component.previewSrc).toBe("data:image/jpeg;base64,AAA"); + }); + + it("returns empty string when formControl has no image", () => { + component.formControl.setValue(""); + expect(component.previewSrc).toBe(""); + }); + + it("returns empty string for server path", () => { + component.formControl.setValue("/uploads/photo.png"); + expect(component.previewSrc).toBe(""); + }); + }); + + describe("displayFileName", () => { + it("returns empty string when no image and no fileName", () => { + expect(component.displayFileName).toBe(""); + }); + + it("returns 'Uploaded image' when image present but no fileName", () => { + component.formControl.setValue("data:image/jpeg;base64,AAA"); + expect(component.displayFileName).toBe("Uploaded image"); + }); + + it("returns fileName when set, even without image", () => { + component.fileName = "photo.jpg"; + expect(component.displayFileName).toBe("photo.jpg"); + }); + }); + + // ── Shared mocking helpers ────────────────────────────────────────────── + // These helpers let us drive the private compressImage / renderCompressedDataUrl + // pipeline end-to-end through onFileSelected. + + function makeFileInput(file?: File): HTMLInputElement { + const input = document.createElement("input"); + input.type = "file"; + if (file) { + Object.defineProperty(input, "files", { + value: [file] as unknown as FileList, + configurable: true, + }); + } + return input; + } + + interface CompressionMockOptions { + /** Value returned by FileReader.result. Default: a valid data URL. */ + readerResult?: string | ArrayBuffer | null; + /** If true, FileReader fires onerror instead of onload. */ + readerError?: boolean; + /** If true, Image fires onerror instead of onload. */ + imageError?: boolean; + /** Image natural dimensions. Default: 100 x 100. */ + imageWidth?: number; + imageHeight?: number; + /** Value(s) returned by canvas.toDataURL on each call. Cycles if shorter than call count. */ + canvasDataUrls?: string[]; + /** If true, canvas.getContext returns null. */ + nullCanvasContext?: boolean; + } + + /** + * Installs fake FileReader, Image, and canvas stubs so the compression + * pipeline runs synchronously via microtasks. Returns a teardown function. + */ + function installCompressionMocks(opts: CompressionMockOptions = {}): () => void { + const savedFileReader = globalThis.FileReader; + const savedImage = globalThis.Image; + + const readerResult = "readerResult" in opts ? opts.readerResult : "data:image/jpeg;base64,AAAA"; + + class FakeFileReader { + onload: ((e: Event) => void) | null = null; + onerror: ((e: Event) => void) | null = null; + result: string | ArrayBuffer | null = readerResult!; + readAsDataURL() { + queueMicrotask(() => { + if (opts.readerError) { + this.onerror?.(new Event("error")); + } else { + this.onload?.(new Event("load")); + } + }); + } + } + (globalThis as any).FileReader = FakeFileReader; + + class FakeImage { + onload: (() => void) | null = null; + onerror: (() => void) | null = null; + width = opts.imageWidth ?? 100; + height = opts.imageHeight ?? 100; + set src(_: string) { + queueMicrotask(() => { + if (opts.imageError) { + this.onerror?.(); + } else { + this.onload?.(); + } + }); + } + } + (globalThis as any).Image = FakeImage; + + let toDataUrlCallIndex = 0; + const canvasDataUrls = opts.canvasDataUrls ?? ["data:image/jpeg;base64,SMALL"]; + + const origCreateElement = document.createElement.bind(document); + vi.spyOn(document, "createElement").mockImplementation((tag: string) => { + if (tag === "canvas") { + return { + width: 0, + height: 0, + getContext: () => + opts.nullCanvasContext + ? null + : { + drawImage: () => {}, + }, + toDataURL: () => { + const url = canvasDataUrls[toDataUrlCallIndex % canvasDataUrls.length]; + toDataUrlCallIndex++; + return url; + }, + } as any; + } + return origCreateElement(tag); + }); + + return () => { + (globalThis as any).FileReader = savedFileReader; + (globalThis as any).Image = savedImage; + vi.restoreAllMocks(); + }; + } + + // ── Successful upload ───────────────────────────────────────────────── + + describe("successful upload", () => { + it("sets fileName, formControl, model, and marks dirty/touched", async () => { + const teardown = installCompressionMocks(); + try { + const imgFile = new File(["fake"], "sunset.jpg", { type: "image/jpeg" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + + expect(component.fileName).toBe("sunset.jpg"); + expect(component.formControl.value).toBe("data:image/jpeg;base64,SMALL"); + expect(component.hasImage).toBe(true); + expect(component.formControl.dirty).toBe(true); + expect(component.formControl.touched).toBe(true); + expect((component.model as any).image).toBe("data:image/jpeg;base64,SMALL"); + expect(component.errorMessage).toBe(""); + } finally { + teardown(); + } + }); + + it("clears a previous error on successful upload", async () => { + component.errorMessage = "previous failure"; + const teardown = installCompressionMocks(); + try { + const imgFile = new File(["fake"], "ok.png", { type: "image/png" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.errorMessage).toBe(""); + } finally { + teardown(); + } + }); + }); + + // ── compressImage rejection paths ───────────────────────────────────── + + describe("compressImage rejection paths", () => { + it("rejects when FileReader.result is not a string (ArrayBuffer)", async () => { + const teardown = installCompressionMocks({ readerResult: new ArrayBuffer(8) }); + try { + const imgFile = new File(["fake"], "pic.png", { type: "image/png" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.errorMessage).toBe("Could not prepare this image. Try a smaller image file."); + expect(component.hasImage).toBe(false); + } finally { + teardown(); + } + }); + + it("rejects when FileReader.result is null", async () => { + const teardown = installCompressionMocks({ readerResult: null }); + try { + const imgFile = new File(["fake"], "pic.png", { type: "image/png" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.errorMessage).toBe("Could not prepare this image. Try a smaller image file."); + } finally { + teardown(); + } + }); + + it("rejects when Image fires onerror", async () => { + const teardown = installCompressionMocks({ imageError: true }); + try { + const imgFile = new File(["fake"], "corrupt.jpg", { type: "image/jpeg" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.errorMessage).toBe("Could not prepare this image. Try a smaller image file."); + expect(component.hasImage).toBe(false); + } finally { + teardown(); + } + }); + }); + + // ── renderCompressedDataUrl edge cases ──────────────────────────────── + + describe("renderCompressedDataUrl (via onFileSelected)", () => { + it("returns empty when canvas context is null", async () => { + const teardown = installCompressionMocks({ nullCanvasContext: true }); + try { + const imgFile = new File(["fake"], "pic.png", { type: "image/png" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + // bestDataUrl starts as "" → compressed is "" → fails startsWith check → rejects + expect(component.errorMessage).toBe("Could not prepare this image. Try a smaller image file."); + } finally { + teardown(); + } + }); + + it("accepts a data URL that fits within the size limit on first attempt", async () => { + const teardown = installCompressionMocks({ + canvasDataUrls: ["data:image/jpeg;base64,FIT"], + }); + try { + const imgFile = new File(["fake"], "small.jpg", { type: "image/jpeg" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.formControl.value).toBe("data:image/jpeg;base64,FIT"); + expect(component.errorMessage).toBe(""); + } finally { + teardown(); + } + }); + + it("reduces quality when the first toDataURL result exceeds the size limit", async () => { + // First call returns oversized URL, second call returns a small one + const oversized = "data:image/jpeg;base64," + "A".repeat(50000); + const small = "data:image/jpeg;base64,OK"; + const teardown = installCompressionMocks({ + canvasDataUrls: [oversized, small], + }); + try { + const imgFile = new File(["fake"], "large.jpg", { type: "image/jpeg" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.formControl.value).toBe(small); + expect(component.errorMessage).toBe(""); + } finally { + teardown(); + } + }); + + it("reduces dimension when quality loop alone is not enough", async () => { + // Quality has 5 steps (0.75, 0.65, 0.55, 0.45, 0.35). + // After exhausting quality at the first maxDimension, it shrinks dimension and retries. + // We make the first 5 calls oversized (quality loop at dim=512), then return small on the 6th (dim=384). + const oversized = "data:image/jpeg;base64," + "A".repeat(50000); + const small = "data:image/jpeg;base64,SHRUNK"; + const teardown = installCompressionMocks({ + canvasDataUrls: [oversized, oversized, oversized, oversized, oversized, small], + }); + try { + const imgFile = new File(["fake"], "huge.jpg", { type: "image/jpeg" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.formControl.value).toBe(small); + } finally { + teardown(); + } + }); + + it("rejects when compressed result never fits within size limit", async () => { + // All toDataURL calls return oversized results + const oversized = "data:image/jpeg;base64," + "A".repeat(50000); + const teardown = installCompressionMocks({ + canvasDataUrls: [oversized], + }); + try { + const imgFile = new File(["fake"], "enormous.jpg", { type: "image/jpeg" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.errorMessage).toBe("Could not prepare this image. Try a smaller image file."); + } finally { + teardown(); + } + }); + + it("rejects when toDataURL returns a non-image data URL", async () => { + const teardown = installCompressionMocks({ + canvasDataUrls: ["data:text/plain;base64,broken"], + }); + try { + const imgFile = new File(["fake"], "weird.png", { type: "image/png" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.errorMessage).toBe("Could not prepare this image. Try a smaller image file."); + } finally { + teardown(); + } + }); + + it("handles a 1x1 pixel image without error", async () => { + const teardown = installCompressionMocks({ + imageWidth: 1, + imageHeight: 1, + canvasDataUrls: ["data:image/jpeg;base64,TINY"], + }); + try { + const imgFile = new File(["fake"], "pixel.png", { type: "image/png" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.formControl.value).toBe("data:image/jpeg;base64,TINY"); + expect(component.errorMessage).toBe(""); + } finally { + teardown(); + } + }); + + it("handles a very wide image (extreme aspect ratio)", async () => { + const teardown = installCompressionMocks({ + imageWidth: 4000, + imageHeight: 10, + canvasDataUrls: ["data:image/jpeg;base64,WIDE"], + }); + try { + const imgFile = new File(["fake"], "banner.jpg", { type: "image/jpeg" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + expect(component.formControl.value).toBe("data:image/jpeg;base64,WIDE"); + expect(component.errorMessage).toBe(""); + } finally { + teardown(); + } + }); + }); + + // ── clearImage edge cases ───────────────────────────────────────────── + + describe("clearImage edge cases", () => { + it("does not update model when key is not a string", () => { + const model: Record<string, unknown> = { someKey: "value" }; + component.field = { + props: {}, + formControl: component.formControl, + key: 42 as any, + model, + } as any; + component.formControl.setValue("data:image/jpeg;base64,AAA"); + + const input = document.createElement("input"); + component.clearImage(input); + + expect(component.formControl.value).toBe(""); + expect(model[42 as any]).toBeUndefined(); + }); + + it("marks formControl dirty and touched", () => { + const input = document.createElement("input"); + component.clearImage(input); + expect(component.formControl.dirty).toBe(true); + expect(component.formControl.touched).toBe(true); + }); + }); + + // ── Consecutive uploads ─────────────────────────────────────────────── + + describe("consecutive uploads", () => { + it("replaces previous upload value with new upload", async () => { + const teardown = installCompressionMocks({ + canvasDataUrls: ["data:image/jpeg;base64,FIRST"], + }); + try { + const firstFile = new File(["a"], "first.jpg", { type: "image/jpeg" }); + const input1 = makeFileInput(firstFile); + await component.onFileSelected({ target: input1 } as unknown as Event); + expect(component.formControl.value).toBe("data:image/jpeg;base64,FIRST"); + expect(component.fileName).toBe("first.jpg"); + } finally { + teardown(); + } + + const teardown2 = installCompressionMocks({ + canvasDataUrls: ["data:image/jpeg;base64,SECOND"], + }); + try { + const secondFile = new File(["b"], "second.png", { type: "image/png" }); + const input2 = makeFileInput(secondFile); + await component.onFileSelected({ target: input2 } as unknown as Event); + expect(component.formControl.value).toBe("data:image/jpeg;base64,SECOND"); + expect(component.fileName).toBe("second.png"); + } finally { + teardown2(); + } + }); + }); + + // ── onFileSelected model update ─────────────────────────────────────── + + describe("onFileSelected model update", () => { + it("does not update model when key is not a string", async () => { + const model: Record<string, unknown> = {}; + component.field = { + props: {}, + formControl: component.formControl, + key: 123 as any, + model, + } as any; + + const teardown = installCompressionMocks(); + try { + const imgFile = new File(["fake"], "pic.png", { type: "image/png" }); + const input = makeFileInput(imgFile); + await component.onFileSelected({ target: input } as unknown as Event); + + // formControl should be updated but model should NOT have the key + expect(component.formControl.value).toBe("data:image/jpeg;base64,SMALL"); + expect(model[123 as any]).toBeUndefined(); + } finally { + teardown(); + } + }); + }); }); diff --git a/frontend/src/app/workspace/component/hugging-face/hugging-face.component.spec.ts b/frontend/src/app/workspace/component/hugging-face/hugging-face.component.spec.ts index 3e2e5bca72..b85e71653e 100644 --- a/frontend/src/app/workspace/component/hugging-face/hugging-face.component.spec.ts +++ b/frontend/src/app/workspace/component/hugging-face/hugging-face.component.spec.ts @@ -661,4 +661,222 @@ describe("HuggingFaceComponent (TestBed)", () => { expect(() => component.ngOnDestroy()).not.toThrow(); }); }); + + // ── Pagination edge cases ── + + describe("pagination edge cases", () => { + it("prevPage at page 0 should keep currentPage at 0", () => { + initComponent("text-generation", buildModels(120)); + + expect(component.currentPage).toBe(0); + component.prevPage(); + expect(component.currentPage).toBe(0); + component.prevPage(); + expect(component.currentPage).toBe(0); + expect(component.pagedModels[0].id).toBe("model/model-0"); + }); + + it("goToPage(0) on empty list should not throw", () => { + initComponent("text-generation", []); + expect(() => component.goToPage(0)).not.toThrow(); + expect(component.currentPage).toBe(0); + expect(component.pagedModels.length).toBe(0); + }); + }); + + // ── Task state snapshot edge cases ── + + describe("task state snapshots", () => { + it("should reset task-scoped fields to defaults on first visit to a new task", () => { + const { field, formGroup } = buildFieldWithFormGroup("text-generation"); + component.field = field; + fixture.detectChanges(); + + http.expectOne(`${API}/huggingface/tasks`).flush(buildTaskResponse()); + http.expectOne(req => req.url.startsWith(`${API}/huggingface/models`)).flush([]); + + // Set non-default values on text-generation + formGroup.get("systemPrompt")!.setValue("Custom prompt"); + formGroup.get("maxNewTokens")!.setValue(512); + formGroup.get("temperature")!.setValue(0.9); + + // Switch to image-classification (first visit) + component.onTaskSelected("image-classification"); + http.expectOne(`${API}/huggingface/models?task=image-classification`).flush([]); + + // First visit defaults should be applied + expect(formGroup.get("systemPrompt")!.value).toBe("You are a helpful assistant."); + expect(formGroup.get("maxNewTokens")!.value).toBe(256); + expect(formGroup.get("temperature")!.value).toBe(0.7); + }); + + it("should preserve task state across multiple switches", () => { + const { field, formGroup } = buildFieldWithFormGroup("text-generation"); + component.field = field; + fixture.detectChanges(); + + http.expectOne(`${API}/huggingface/tasks`).flush(buildTaskResponse()); + http.expectOne(req => req.url.startsWith(`${API}/huggingface/models`)).flush([]); + + // Set values on text-generation + formGroup.get("promptColumn")!.setValue("prompt_col"); + formGroup.get("modelId")!.setValue("my-org/my-model"); + + // Switch away + component.onTaskSelected("image-classification"); + http.expectOne(`${API}/huggingface/models?task=image-classification`).flush([]); + + // Set values on image-classification + formGroup.get("modelId")!.setValue("img-org/img-model"); + + // Switch back to text-generation + component.onTaskSelected("text-generation"); + expect(formGroup.get("promptColumn")!.value).toBe("prompt_col"); + expect(formGroup.get("modelId")!.value).toBe("my-org/my-model"); + + // Switch back to image-classification + component.onTaskSelected("image-classification"); + http.match(`${API}/huggingface/models?task=image-classification`); // might be cached + expect(formGroup.get("modelId")!.value).toBe("img-org/img-model"); + }); + }); + + // ── Server-side search edge cases ── + + describe("server search edge cases", () => { + it("should handle server search error gracefully", fakeAsync(() => { + const { field } = buildFieldWithFormGroup(); + component.field = field; + fixture.detectChanges(); + + http.expectOne(`${API}/huggingface/tasks`).flush(buildTaskResponse()); + const modelsReq = http.expectOne(req => req.url.startsWith(`${API}/huggingface/models`)); + modelsReq.flush(buildModels(5), { headers: { "X-Texera-Truncated": "true" } }); + + component.onSearchInput("fail-query"); + tick(300); + + const searchReq = http.expectOne(req => req.url.includes("search=fail-query")); + searchReq.error(new ProgressEvent("error")); + + // Should not crash; searchLoading should be reset + expect(component.searchLoading).toBe(false); + })); + + it("should replace search results when a new query supersedes the previous one", fakeAsync(() => { + const { field } = buildFieldWithFormGroup(); + component.field = field; + fixture.detectChanges(); + + http.expectOne(`${API}/huggingface/tasks`).flush(buildTaskResponse()); + const modelsReq = http.expectOne(req => req.url.startsWith(`${API}/huggingface/models`)); + modelsReq.flush(buildModels(5), { headers: { "X-Texera-Truncated": "true" } }); + + // First search — complete it + component.onSearchInput("query-1"); + tick(300); + const req1 = http.expectOne(req => req.url.includes("search=query-1")); + req1.flush([{ id: "result-1", label: "result-1" }]); + + expect(component.pagedModels.length).toBe(1); + expect(component.pagedModels[0].id).toBe("result-1"); + + // Second search — results should replace the first + component.onSearchInput("query-2"); + tick(300); + const req2 = http.expectOne(req => req.url.includes("search=query-2")); + req2.flush([ + { id: "result-2a", label: "result-2a" }, + { id: "result-2b", label: "result-2b" }, + ]); + + expect(component.pagedModels.length).toBe(2); + expect(component.pagedModels[0].id).toBe("result-2a"); + })); + }); + + // ── getCurrentTaskTag fallbacks ── + + describe("getCurrentTaskTag", () => { + it("should read task from model.task when available", () => { + const { field } = buildFieldWithFormGroup("image-classification"); + component.field = field; + fixture.detectChanges(); + + http.expectOne(`${API}/huggingface/tasks`).flush(buildTaskResponse()); + http.expectOne(`${API}/huggingface/models?task=image-classification`).flush(buildModels(2, "img")); + + expect(component.selectedTaskTag).toBe("image-classification"); + }); + + it("should read task from formControl.parent when model.task is empty", () => { + const { field, formGroup } = buildFieldWithFormGroup(""); + // Clear model.task but set parent form control + field.model!["task"] = ""; + formGroup.get("task")!.setValue("summarization"); + component.field = field; + fixture.detectChanges(); + + http.expectOne(`${API}/huggingface/tasks`).flush(buildTaskResponse()); + http.expectOne(`${API}/huggingface/models?task=summarization`).flush([]); + + expect(component.selectedTaskTag).toBe("summarization"); + }); + }); + + // ── Model selection edge cases ── + + describe("model selection edge cases", () => { + it("onModelSelected should mark formControl as dirty", () => { + const { field } = buildFieldWithFormGroup(); + component.field = field; + fixture.detectChanges(); + + http.expectOne(`${API}/huggingface/tasks`).flush(buildTaskResponse()); + http.expectOne(req => req.url.startsWith(`${API}/huggingface/models`)).flush(buildModels(3)); + + component.onModelSelected("model/model-2"); + expect(field.formControl!.value).toBe("model/model-2"); + }); + + it("onModelSelected should overwrite previous selection", () => { + const { field } = buildFieldWithFormGroup(); + component.field = field; + fixture.detectChanges(); + + http.expectOne(`${API}/huggingface/tasks`).flush(buildTaskResponse()); + http.expectOne(req => req.url.startsWith(`${API}/huggingface/models`)).flush(buildModels(3)); + + component.onModelSelected("model/model-0"); + expect(field.formControl!.value).toBe("model/model-0"); + + component.onModelSelected("model/model-2"); + expect(field.formControl!.value).toBe("model/model-2"); + }); + }); + + // ── retryLoad ── + + describe("retryLoad", () => { + it("should clear the cached error and refetch models", () => { + const { field } = buildFieldWithFormGroup(); + component.field = field; + fixture.detectChanges(); + + http.expectOne(`${API}/huggingface/tasks`).flush(buildTaskResponse()); + http.expectOne(req => req.url.startsWith(`${API}/huggingface/models`)).error(new ProgressEvent("error")); + + expect(component.errorMessage).toBeTruthy(); + + // Retry should clear error and fetch again + component.retryLoad(); + expect(component.errorMessage).toBeNull(); + + const retryReq = http.expectOne(req => req.url.startsWith(`${API}/huggingface/models`)); + retryReq.flush(buildModels(5)); + + expect(component.pagedModels.length).toBe(5); + expect(component.loading).toBe(false); + }); + }); }); diff --git a/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.html b/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.html index 3cdd88911a..de255386ae 100644 --- a/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.html +++ b/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.html @@ -103,6 +103,88 @@ *ngIf="formlyFields && formlyFormGroup" [formGroup]="formlyFormGroup" class="property-editor-form"> + <div + *ngIf="huggingFaceTaskPreview as preview" + class="hf-task-preview"> + <p class="hf-task-preview-label">Task Preview:</p> + <div class="hf-task-preview-card"> + <ng-container [ngSwitch]="preview.kind"> + <video + *ngSwitchCase="'video'" + autoplay + loop + [muted]="true" + playsinline + controls + class="hf-task-preview-media" + [src]="preview.assetSrc"></video> + <img + *ngSwitchCase="'image'" + class="hf-task-preview-media" + [src]="preview.assetSrc" + alt="Hugging Face task preview" /> + <audio + *ngSwitchCase="'audio'" + controls + preload="metadata" + class="hf-task-preview-audio" + [src]="preview.assetSrc"></audio> + <div + *ngSwitchCase="'text'" + class="hf-task-preview-text-surface"> + <div class="hf-task-preview-text-title">{{ preview.title }}</div> + <div class="hf-task-preview-text-body">{{ preview.body }}</div> + <div + *ngIf="preview.outputBody" + class="hf-task-preview-text-output"> + {{ preview.outputBody }} + </div> + </div> + </ng-container> + + <div class="hf-task-preview-meta"> + <div class="hf-task-preview-title">{{ preview.title }}</div> + <div + *ngIf="preview.inputLabel || preview.outputLabel" + class="hf-task-preview-flow"> + <span + *ngIf="preview.inputLabel" + class="hf-task-preview-chip" + >{{ preview.inputLabel }}</span + > + <span + *ngIf="preview.outputLabel" + class="hf-task-preview-arrow" + >→</span + > + <span + *ngIf="preview.outputLabel" + class="hf-task-preview-chip" + >{{ preview.outputLabel }}</span + > + </div> + <div + *ngIf="preview.body && preview.kind !== 'text'" + class="hf-task-preview-description"> + {{ preview.body }} + </div> + <div + *ngIf="preview.outputBody && preview.kind !== 'text'" + class="hf-task-preview-output"> + {{ preview.outputBody }} + </div> + <div + *ngIf="preview.pills?.length" + class="hf-task-preview-pills"> + <span + *ngFor="let pill of preview.pills" + class="hf-task-preview-pill" + >{{ pill }}</span + > + </div> + </div> + </div> + </div> <formly-form (modelChange)="onFormChanges($event)" [fields]="formlyFields" diff --git a/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.scss b/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.scss index 0cd197f7a5..0c65243884 100644 --- a/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.scss +++ b/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.scss @@ -78,3 +78,130 @@ -webkit-box-orient: vertical; } } + +.hf-task-preview { + margin-bottom: 20px; +} + +.hf-task-preview-label { + font-weight: 700; + margin-bottom: 8px; + text-align: center; +} + +.hf-task-preview-card { + border: 1px solid #d9d9d9; + border-radius: 6px; + padding: 12px; + background: #fafafa; + display: flex; + flex-direction: column; + gap: 12px; + align-items: center; +} + +.hf-task-preview-media { + width: 100%; + max-width: 250px; + border-radius: 4px; + border: 1px solid #ccc; + display: block; +} + +.hf-task-preview-audio { + width: 100%; + max-width: 250px; + display: block; +} + +.hf-task-preview-meta { + width: 100%; + display: flex; + flex-direction: column; + gap: 8px; + align-items: center; + text-align: center; +} + +.hf-task-preview-title { + font-size: 13px; + font-weight: 600; + color: rgba(0, 0, 0, 0.88); +} + +.hf-task-preview-flow { + display: flex; + flex-wrap: wrap; + justify-content: center; + align-items: center; + gap: 6px; +} + +.hf-task-preview-chip, +.hf-task-preview-pill { + display: inline-flex; + align-items: center; + border: 1px solid #d9d9d9; + border-radius: 999px; + background: #fff; + padding: 2px 8px; + font-size: 11px; + color: rgba(0, 0, 0, 0.72); +} + +.hf-task-preview-arrow { + font-size: 12px; + color: rgba(0, 0, 0, 0.45); +} + +.hf-task-preview-description, +.hf-task-preview-output { + font-size: 12px; + line-height: 1.5; + color: rgba(0, 0, 0, 0.65); +} + +.hf-task-preview-output { + padding: 8px 10px; + background: #fff; + border: 1px solid #e8e8e8; + border-radius: 4px; + width: 100%; +} + +.hf-task-preview-pills { + display: flex; + flex-wrap: wrap; + justify-content: center; + gap: 6px; +} + +.hf-task-preview-text-surface { + width: 100%; + border: 1px solid #d9d9d9; + border-radius: 4px; + background: #fff; + padding: 12px; + display: flex; + flex-direction: column; + gap: 8px; +} + +.hf-task-preview-text-title { + font-size: 13px; + font-weight: 600; + color: rgba(0, 0, 0, 0.88); +} + +.hf-task-preview-text-body, +.hf-task-preview-text-output { + font-size: 12px; + line-height: 1.5; + color: rgba(0, 0, 0, 0.72); + white-space: pre-wrap; +} + +.hf-task-preview-text-output { + padding-top: 8px; + border-top: 1px solid #f0f0f0; +} diff --git a/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.spec.ts b/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.spec.ts index eb0088ba18..896e64b628 100644 --- a/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.spec.ts +++ b/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.spec.ts @@ -28,10 +28,11 @@ import { DatePipe } from "@angular/common"; import { By } from "@angular/platform-browser"; import { BrowserAnimationsModule } from "@angular/platform-browser/animations"; import { FormsModule, ReactiveFormsModule } from "@angular/forms"; -import { FormlyModule } from "@ngx-formly/core"; +import { FormlyFieldConfig, FormlyModule } from "@ngx-formly/core"; import { TEXERA_FORMLY_CONFIG } from "../../../../common/formly/formly-config"; import { HttpClientTestingModule } from "@angular/common/http/testing"; import { + mockHuggingFacePredicate, mockPoint, mockResultPredicate, mockScanPredicate, @@ -337,4 +338,1189 @@ describe("OperatorPropertyEditFrameComponent", () => { expect(descEl).toBeNull(); }); }); + + // ── HuggingFace task-aware visibility tests ── + + it("should return null huggingFaceTaskPreview for non-HF operators", () => { + workflowActionService.addOperator(mockScanPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, mockScanPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview).toBeNull(); + }); + + it("should return a task preview for HuggingFace operator with a known task", () => { + workflowActionService.addOperator(mockHuggingFacePredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, mockHuggingFacePredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.kind).toBe("text"); + expect(preview!.title).toBe("Text generation preview"); + }); + + it("should return a fallback preview for HuggingFace operator with an unknown task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "some-unknown-task", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.kind).toBe("text"); + expect(preview!.title).toBe("Some Unknown Task"); + }); + + it("should return image kind preview for image-classification task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "image-classification", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.kind).toBe("image"); + }); + + it("should return audio kind preview for text-to-speech task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "text-to-speech", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.kind).toBe("audio"); + }); + + it("should return video kind preview for text-to-video task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "text-to-video", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.kind).toBe("video"); + }); + + it("should return null preview when HuggingFace task is empty", () => { + const hfPredicate = { ...cloneDeep(mockHuggingFacePredicate), operatorProperties: { task: "", modelId: "" } }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview).toBeNull(); + }); + + // ── HuggingFace field visibility and validator tests ── + + function getHfField(key: string): FormlyFieldConfig | undefined { + return component.formlyFields?.[0]?.fieldGroup?.find(f => f.key === key); + } + + let currentTask: string = ""; + + let hfOperatorCounter = 0; + + function initHfOperator(task: string): void { + currentTask = task; + hfOperatorCounter++; + const pred = { + ...cloneDeep(mockHuggingFacePredicate), + operatorID: `hf-test-${hfOperatorCounter}`, + operatorProperties: { task, modelId: "org/model" }, + }; + workflowActionService.addOperator(pred, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, pred.operatorID, true), + }); + fixture.detectChanges(); + } + + function evalHide(field: FormlyFieldConfig | undefined): boolean { + if (!field || !field.expressions) return false; + const hideFn = (field.expressions as Record<string, Function>)["hide"]; + if (!hideFn) return !!field.hide; + // Provide model context so getSelectedTask can find the task + const fieldWithModel = { ...field, model: { task: currentTask } } as FormlyFieldConfig; + return hideFn(fieldWithModel); + } + + it("should hide imageInput for text-generation task", () => { + initHfOperator("text-generation"); + expect(evalHide(getHfField("imageInput"))).toBe(true); + }); + + it("should show imageInput for image-classification task", () => { + initHfOperator("image-classification"); + expect(evalHide(getHfField("imageInput"))).toBe(false); + }); + + it("should hide audioInput for text-generation task", () => { + initHfOperator("text-generation"); + expect(evalHide(getHfField("audioInput"))).toBe(true); + }); + + it("should show audioInput for automatic-speech-recognition task", () => { + initHfOperator("automatic-speech-recognition"); + expect(evalHide(getHfField("audioInput"))).toBe(false); + }); + + it("should hide promptColumn for image-only tasks", () => { + initHfOperator("image-classification"); + expect(evalHide(getHfField("promptColumn"))).toBe(true); + }); + + it("should hide promptColumn for audio-only tasks", () => { + initHfOperator("automatic-speech-recognition"); + expect(evalHide(getHfField("promptColumn"))).toBe(true); + }); + + it("should show promptColumn for text-generation task", () => { + initHfOperator("text-generation"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show systemPrompt only for text-generation", () => { + initHfOperator("text-generation"); + expect(evalHide(getHfField("systemPrompt"))).toBe(false); + + initHfOperator("image-classification"); + expect(evalHide(getHfField("systemPrompt"))).toBe(true); + }); + + it("should show contextColumn only for question-answering", () => { + initHfOperator("question-answering"); + expect(evalHide(getHfField("contextColumn"))).toBe(false); + + initHfOperator("text-generation"); + expect(evalHide(getHfField("contextColumn"))).toBe(true); + }); + + it("should show candidateLabels only for classification tasks", () => { + initHfOperator("zero-shot-classification"); + expect(evalHide(getHfField("candidateLabels"))).toBe(false); + + initHfOperator("text-generation"); + expect(evalHide(getHfField("candidateLabels"))).toBe(true); + }); + + it("requiredPromptColumn validator should pass when not a prompt-required task", () => { + initHfOperator("image-classification"); + const field = getHfField("promptColumn"); + const validator = field?.validators?.["requiredPromptColumn"]; + expect(validator).toBeDefined(); + const mockField = { ...field, model: { task: "image-classification", promptColumn: "" } } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + it("requiredPromptColumn validator should fail when prompt-required task has no column", () => { + initHfOperator("text-generation"); + const field = getHfField("promptColumn"); + const validator = field?.validators?.["requiredPromptColumn"]; + expect(validator).toBeDefined(); + const mockField = { ...field, model: { task: "text-generation", promptColumn: "" } } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(false); + }); + + it("requiredImageInput validator should pass when not an image task", () => { + initHfOperator("text-generation"); + const field = getHfField("imageInput"); + const validator = field?.validators?.["requiredImageInput"]; + expect(validator).toBeDefined(); + const mockField = { ...field, model: { task: "text-generation", imageInput: "" } } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + it("requiredAudioInput validator should pass when not an audio task", () => { + initHfOperator("text-generation"); + const field = getHfField("audioInput"); + const validator = field?.validators?.["requiredAudioInput"]; + expect(validator).toBeDefined(); + const mockField = { ...field, model: { task: "text-generation", audioInput: "" } } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + // ── Additional field visibility tests ── + + it("should show sentencesColumn only for sentence-similarity and text-ranking", () => { + initHfOperator("sentence-similarity"); + expect(evalHide(getHfField("sentencesColumn"))).toBe(false); + + initHfOperator("text-ranking"); + expect(evalHide(getHfField("sentencesColumn"))).toBe(false); + + initHfOperator("text-generation"); + expect(evalHide(getHfField("sentencesColumn"))).toBe(true); + }); + + it("should show inputImageColumn for image tasks", () => { + initHfOperator("image-classification"); + expect(evalHide(getHfField("inputImageColumn"))).toBe(false); + + initHfOperator("text-generation"); + expect(evalHide(getHfField("inputImageColumn"))).toBe(true); + }); + + it("should show inputAudioColumn for audio tasks", () => { + initHfOperator("automatic-speech-recognition"); + expect(evalHide(getHfField("inputAudioColumn"))).toBe(false); + + initHfOperator("text-generation"); + expect(evalHide(getHfField("inputAudioColumn"))).toBe(true); + }); + + it("should hide maxNewTokens and temperature for non-text-generation tasks", () => { + initHfOperator("image-classification"); + expect(evalHide(getHfField("maxNewTokens"))).toBe(true); + expect(evalHide(getHfField("temperature"))).toBe(true); + + initHfOperator("text-generation"); + expect(evalHide(getHfField("maxNewTokens"))).toBe(false); + expect(evalHide(getHfField("temperature"))).toBe(false); + }); + + it("should show candidateLabels for zero-shot-image-classification", () => { + initHfOperator("zero-shot-image-classification"); + expect(evalHide(getHfField("candidateLabels"))).toBe(false); + }); + + // ── Additional validator edge-case tests ── + + it("requiredPromptColumn validator should pass when prompt-required task has a column", () => { + initHfOperator("text-generation"); + const field = getHfField("promptColumn"); + const validator = field?.validators?.["requiredPromptColumn"]; + const mockField = { ...field, model: { task: "text-generation", promptColumn: "text_col" } } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + it("requiredImageInput validator should fail when image task has no image and no column", () => { + initHfOperator("image-classification"); + const field = getHfField("imageInput"); + const validator = field?.validators?.["requiredImageInput"]; + const mockField = { + ...field, + model: { task: "image-classification", imageInput: "", inputImageColumn: "" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(false); + }); + + it("requiredImageInput validator should pass when image task has inputImageColumn set", () => { + initHfOperator("image-classification"); + const field = getHfField("imageInput"); + const validator = field?.validators?.["requiredImageInput"]; + const mockField = { + ...field, + model: { task: "image-classification", imageInput: "", inputImageColumn: "img_col" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + it("requiredImageInput validator should pass when image task has image uploaded", () => { + initHfOperator("image-classification"); + const field = getHfField("imageInput"); + const validator = field?.validators?.["requiredImageInput"]; + const mockField = { + ...field, + model: { task: "image-classification", imageInput: "/tmp/img.png", inputImageColumn: "" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + it("requiredAudioInput validator should fail when audio task has no audio and no column", () => { + initHfOperator("automatic-speech-recognition"); + const field = getHfField("audioInput"); + const validator = field?.validators?.["requiredAudioInput"]; + const mockField = { + ...field, + model: { task: "automatic-speech-recognition", audioInput: "", inputAudioColumn: "" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(false); + }); + + it("requiredAudioInput validator should pass when audio task has inputAudioColumn set", () => { + initHfOperator("automatic-speech-recognition"); + const field = getHfField("audioInput"); + const validator = field?.validators?.["requiredAudioInput"]; + const mockField = { + ...field, + model: { task: "automatic-speech-recognition", audioInput: "", inputAudioColumn: "audio_col" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + it("requiredAudioInput validator should pass when audio task has audio uploaded", () => { + initHfOperator("automatic-speech-recognition"); + const field = getHfField("audioInput"); + const validator = field?.validators?.["requiredAudioInput"]; + const mockField = { + ...field, + model: { task: "automatic-speech-recognition", audioInput: "/tmp/clip.wav", inputAudioColumn: "" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + // ── HuggingFace task preview additional tests ── + + it("should return image kind preview for visual-question-answering task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "visual-question-answering", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.kind).toBe("image"); + }); + + it("should return text kind preview for question-answering task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "question-answering", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.kind).toBe("text"); + }); + + it("should hide the task field for HuggingFace operators", () => { + initHfOperator("text-generation"); + const taskField = getHfField("task"); + expect(taskField?.hide).toBe(true); + }); + + // ── Field type assignments ── + + it("should set modelId field type to 'huggingface' for HF operators", () => { + initHfOperator("text-generation"); + const field = getHfField("modelId"); + expect(field?.type).toBe("huggingface"); + }); + + it("should set imageInput field type to 'huggingface-image-upload'", () => { + initHfOperator("image-classification"); + const field = getHfField("imageInput"); + expect(field?.type).toBe("huggingface-image-upload"); + }); + + it("should set audioInput field type to 'huggingface-audio-upload'", () => { + initHfOperator("automatic-speech-recognition"); + const field = getHfField("audioInput"); + expect(field?.type).toBe("huggingface-audio-upload"); + }); + + // ── Visibility when task is undefined ── + + it("should hide imageInput when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("imageInput"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); + + it("should hide audioInput when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("audioInput"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); + + it("should hide inputImageColumn when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("inputImageColumn"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); + + it("should hide inputAudioColumn when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("inputAudioColumn"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); + + // ── Additional image task visibility ── + + it("should show imageInput for image-to-video task", () => { + initHfOperator("image-to-video"); + expect(evalHide(getHfField("imageInput"))).toBe(false); + }); + + it("should show imageInput for image-to-image task", () => { + initHfOperator("image-to-image"); + expect(evalHide(getHfField("imageInput"))).toBe(false); + }); + + it("should show imageInput for document-question-answering task", () => { + initHfOperator("document-question-answering"); + expect(evalHide(getHfField("imageInput"))).toBe(false); + }); + + it("should show imageInput for image-text-to-text task", () => { + initHfOperator("image-text-to-text"); + expect(evalHide(getHfField("imageInput"))).toBe(false); + }); + + // ── Audio task visibility ── + + it("should show audioInput for audio-classification task", () => { + initHfOperator("audio-classification"); + expect(evalHide(getHfField("audioInput"))).toBe(false); + }); + + it("should show inputAudioColumn for audio-classification task", () => { + initHfOperator("audio-classification"); + expect(evalHide(getHfField("inputAudioColumn"))).toBe(false); + }); + + // ── promptColumn visibility for mixed tasks ── + + it("should show promptColumn for visual-question-answering (image + prompt)", () => { + initHfOperator("visual-question-answering"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for document-question-answering (image + prompt)", () => { + initHfOperator("document-question-answering"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for zero-shot-classification", () => { + initHfOperator("zero-shot-classification"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for summarization", () => { + initHfOperator("summarization"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for translation", () => { + initHfOperator("translation"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + // ── Validator with formControl value ── + + it("requiredImageInput validator should pass when image task has formControl value", () => { + initHfOperator("image-classification"); + const field = getHfField("imageInput"); + const validator = field?.validators?.["requiredImageInput"]; + const mockField = { + ...field, + model: { task: "image-classification", imageInput: "", inputImageColumn: "" }, + formControl: { value: "data:image/png;base64,abc" }, + } as unknown as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + it("requiredAudioInput validator should pass when audio task has formControl value", () => { + initHfOperator("automatic-speech-recognition"); + const field = getHfField("audioInput"); + const validator = field?.validators?.["requiredAudioInput"]; + const mockField = { + ...field, + model: { task: "automatic-speech-recognition", audioInput: "", inputAudioColumn: "" }, + formControl: { value: "/tmp/clip.wav" }, + } as unknown as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + it("requiredPromptColumn validator should pass when formControl has value", () => { + initHfOperator("text-generation"); + const field = getHfField("promptColumn"); + const validator = field?.validators?.["requiredPromptColumn"]; + const mockField = { + ...field, + model: { task: "text-generation", promptColumn: "" }, + formControl: { value: "text_col" }, + } as unknown as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + // ── Additional task preview tests ── + + it("should return audio kind preview for automatic-speech-recognition task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "automatic-speech-recognition", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.kind).toBe("audio"); + }); + + it("should return image kind preview for image-to-image task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "image-to-image", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.kind).toBe("image"); + }); + + it("should return null huggingFaceTaskPreview when operator is deleted", () => { + workflowActionService.addOperator(mockHuggingFacePredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, mockHuggingFacePredicate.operatorID, true), + }); + fixture.detectChanges(); + workflowActionService.deleteOperator(mockHuggingFacePredicate.operatorID); + expect(component.huggingFaceTaskPreview).toBeNull(); + }); + + // ── formatTaskTitle via fallback preview ── + + it("should title-case multi-segment unknown task in fallback preview", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "my-custom-pipeline", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview; + expect(preview).toBeTruthy(); + expect(preview!.title).toBe("My Custom Pipeline"); + }); + + it("should title-case single-word unknown task in fallback preview", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "embeddings", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.title).toBe("Embeddings"); + }); + + // ── Task preview content validation ── + + it("should include assetSrc and pills in image-classification preview", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "image-classification", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview!; + expect(preview.assetSrc).toBe("assets/sample-image.png"); + expect(preview.pills).toEqual(["superhero", "cityscape", "action"]); + expect(preview.inputLabel).toBe("Image input"); + expect(preview.outputLabel).toBe("Predicted labels"); + }); + + it("should include outputBody in text-generation preview", () => { + workflowActionService.addOperator(mockHuggingFacePredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, mockHuggingFacePredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview!; + expect(preview.outputBody).toBeDefined(); + expect(preview.body).toBeDefined(); + expect(preview.inputLabel).toBe("Prompt"); + expect(preview.outputLabel).toBe("Generated text"); + }); + + it("should return video kind preview for image-to-video task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "image-to-video", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview!; + expect(preview.kind).toBe("video"); + expect(preview.assetSrc).toBe("assets/sample-video.mp4"); + }); + + it("should return text kind preview for zero-shot-classification task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "zero-shot-classification", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview!; + expect(preview.kind).toBe("text"); + expect(preview.pills).toEqual(["business", "operations", "support"]); + }); + + it("should return text kind preview for fill-mask task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "fill-mask", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview!; + expect(preview.kind).toBe("text"); + expect(preview.pills).toEqual(["city", "day", "crowd"]); + }); + + it("should return image kind preview for object-detection task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "object-detection", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview!; + expect(preview.kind).toBe("image"); + expect(preview.pills).toEqual(["person", "building", "sky"]); + }); + + it("should return image kind preview for text-to-image task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "text-to-image", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("image"); + }); + + it("should return text kind preview for text-classification task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "text-classification", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview!; + expect(preview.kind).toBe("text"); + expect(preview.pills).toEqual(["positive", "announcement"]); + }); + + it("should return text kind preview for token-classification task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "token-classification", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("text"); + }); + + it("should return text kind preview for table-question-answering task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "table-question-answering", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("text"); + }); + + it("should return text kind preview for feature-extraction task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "feature-extraction", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("text"); + }); + + it("should return image kind preview for image-segmentation task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "image-segmentation", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("image"); + }); + + it("should return image kind preview for image-to-text task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "image-to-text", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + const preview = component.huggingFaceTaskPreview!; + expect(preview.kind).toBe("image"); + expect(preview.outputBody).toBeDefined(); + }); + + it("should return image kind preview for document-question-answering task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "document-question-answering", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("image"); + }); + + it("should return image kind preview for zero-shot-image-classification task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "zero-shot-image-classification", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("image"); + }); + + it("should return image kind preview for image-text-to-text task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "image-text-to-text", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("image"); + }); + + it("should return text kind preview for sentence-similarity task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "sentence-similarity", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("text"); + }); + + it("should return text kind preview for text-ranking task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "text-ranking", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("text"); + }); + + it("should return text kind preview for translation task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "translation", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("text"); + }); + + it("should return text kind preview for summarization task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "summarization", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("text"); + }); + + it("should return audio kind preview for audio-classification task", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: "audio-classification", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview!.kind).toBe("audio"); + }); + + // ── Validator message strings ── + + it("requiredImageInput validator should return correct message", () => { + initHfOperator("image-classification"); + const field = getHfField("imageInput"); + const validator = field?.validators?.["requiredImageInput"]; + expect(validator!.message()).toBe("Upload an image or select an Input Image Column for this task."); + }); + + it("requiredAudioInput validator should return correct message", () => { + initHfOperator("automatic-speech-recognition"); + const field = getHfField("audioInput"); + const validator = field?.validators?.["requiredAudioInput"]; + expect(validator!.message()).toBe("Upload audio or select an Input Audio Column for this task."); + }); + + it("requiredPromptColumn validator should return correct message", () => { + initHfOperator("text-generation"); + const field = getHfField("promptColumn"); + const validator = field?.validators?.["requiredPromptColumn"]; + expect(validator!.message()).toBe("Select a prompt column for this task."); + }); + + // ── Additional promptColumn visibility for remaining tasks ── + + it("should show promptColumn for token-classification", () => { + initHfOperator("token-classification"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for table-question-answering", () => { + initHfOperator("table-question-answering"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for feature-extraction", () => { + initHfOperator("feature-extraction"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for fill-mask", () => { + initHfOperator("fill-mask"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for sentence-similarity", () => { + initHfOperator("sentence-similarity"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for text-ranking", () => { + initHfOperator("text-ranking"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should show promptColumn for image-text-to-text", () => { + initHfOperator("image-text-to-text"); + expect(evalHide(getHfField("promptColumn"))).toBe(false); + }); + + it("should hide promptColumn for object-detection (image-only)", () => { + initHfOperator("object-detection"); + expect(evalHide(getHfField("promptColumn"))).toBe(true); + }); + + it("should hide promptColumn for image-segmentation (image-only)", () => { + initHfOperator("image-segmentation"); + expect(evalHide(getHfField("promptColumn"))).toBe(true); + }); + + it("should hide promptColumn for image-to-text (image-only)", () => { + initHfOperator("image-to-text"); + expect(evalHide(getHfField("promptColumn"))).toBe(true); + }); + + // ── Field visibility for media-generation tasks ── + + it("should hide imageInput for text-to-image task", () => { + initHfOperator("text-to-image"); + expect(evalHide(getHfField("imageInput"))).toBe(true); + }); + + it("should hide imageInput for text-to-speech task", () => { + initHfOperator("text-to-speech"); + expect(evalHide(getHfField("imageInput"))).toBe(true); + }); + + it("should hide audioInput for text-to-image task", () => { + initHfOperator("text-to-image"); + expect(evalHide(getHfField("audioInput"))).toBe(true); + }); + + it("should hide audioInput for text-to-speech task", () => { + initHfOperator("text-to-speech"); + expect(evalHide(getHfField("audioInput"))).toBe(true); + }); + + it("should show imageInput for zero-shot-image-classification task", () => { + initHfOperator("zero-shot-image-classification"); + expect(evalHide(getHfField("imageInput"))).toBe(false); + }); + + it("should show inputImageColumn for zero-shot-image-classification", () => { + initHfOperator("zero-shot-image-classification"); + expect(evalHide(getHfField("inputImageColumn"))).toBe(false); + }); + + it("should show inputImageColumn for image-to-image", () => { + initHfOperator("image-to-image"); + expect(evalHide(getHfField("inputImageColumn"))).toBe(false); + }); + + it("should show inputImageColumn for image-to-video", () => { + initHfOperator("image-to-video"); + expect(evalHide(getHfField("inputImageColumn"))).toBe(false); + }); + + // ── Validator edge cases: zero-shot-image-classification ── + + it("requiredImageInput validator should fail for zero-shot-image-classification with no input", () => { + initHfOperator("zero-shot-image-classification"); + const field = getHfField("imageInput"); + const validator = field?.validators?.["requiredImageInput"]; + const mockField = { + ...field, + model: { task: "zero-shot-image-classification", imageInput: "", inputImageColumn: "" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(false); + }); + + it("requiredImageInput validator should pass for zero-shot-image-classification with column", () => { + initHfOperator("zero-shot-image-classification"); + const field = getHfField("imageInput"); + const validator = field?.validators?.["requiredImageInput"]; + const mockField = { + ...field, + model: { task: "zero-shot-image-classification", imageInput: "", inputImageColumn: "img_col" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(true); + }); + + // ── Validator edge cases: whitespace-only values ── + + it("requiredPromptColumn validator should fail when value is whitespace-only", () => { + initHfOperator("text-generation"); + const field = getHfField("promptColumn"); + const validator = field?.validators?.["requiredPromptColumn"]; + const mockField = { ...field, model: { task: "text-generation", promptColumn: " " } } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(false); + }); + + it("requiredImageInput validator should fail when imageInput is whitespace-only", () => { + initHfOperator("image-classification"); + const field = getHfField("imageInput"); + const validator = field?.validators?.["requiredImageInput"]; + const mockField = { + ...field, + model: { task: "image-classification", imageInput: " ", inputImageColumn: "" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(false); + }); + + it("requiredAudioInput validator should fail when audioInput is whitespace-only", () => { + initHfOperator("automatic-speech-recognition"); + const field = getHfField("audioInput"); + const validator = field?.validators?.["requiredAudioInput"]; + const mockField = { + ...field, + model: { task: "automatic-speech-recognition", audioInput: " ", inputAudioColumn: "" }, + } as FormlyFieldConfig; + expect(validator!.expression(null as any, mockField)).toBe(false); + }); + + // ── getSelectedTask fallback: form.get("task") ── + + it("should use form.get task value for hide expression when model.task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("imageInput"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + // Simulate: model has no task but form has it + const mockField = { + model: {}, + form: { get: (key: string) => (key === "task" ? { value: "image-classification" } : null) }, + } as unknown as FormlyFieldConfig; + expect(hideFn(mockField)).toBe(false); // image-classification is an image task + }); + + it("should use formControl.parent.get task value when model and form are empty", () => { + initHfOperator("text-generation"); + const field = getHfField("audioInput"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + const mockField = { + model: {}, + formControl: { + parent: { get: (key: string) => (key === "task" ? { value: "automatic-speech-recognition" } : null) }, + }, + } as unknown as FormlyFieldConfig; + expect(hideFn(mockField)).toBe(false); // ASR is an audio task + }); + + // ── Null / undefined preview edge cases ── + + it("should return null preview when task is whitespace-only", () => { + const hfPredicate = { + ...cloneDeep(mockHuggingFacePredicate), + operatorProperties: { task: " ", modelId: "" }, + }; + workflowActionService.addOperator(hfPredicate, mockPoint); + component.ngOnChanges({ + currentOperatorId: new SimpleChange(undefined, hfPredicate.operatorID, true), + }); + fixture.detectChanges(); + expect(component.huggingFaceTaskPreview).toBeNull(); + }); + + // ── systemPrompt/maxNewTokens/temperature visibility for more tasks ── + + it("should hide systemPrompt for automatic-speech-recognition", () => { + initHfOperator("automatic-speech-recognition"); + expect(evalHide(getHfField("systemPrompt"))).toBe(true); + expect(evalHide(getHfField("maxNewTokens"))).toBe(true); + expect(evalHide(getHfField("temperature"))).toBe(true); + }); + + it("should hide contextColumn for image-classification", () => { + initHfOperator("image-classification"); + expect(evalHide(getHfField("contextColumn"))).toBe(true); + }); + + it("should hide candidateLabels for text-generation", () => { + initHfOperator("text-generation"); + expect(evalHide(getHfField("candidateLabels"))).toBe(true); + }); + + it("should hide sentencesColumn for question-answering", () => { + initHfOperator("question-answering"); + expect(evalHide(getHfField("sentencesColumn"))).toBe(true); + }); + + // ── Visibility when task is undefined for remaining fields ── + + it("should hide promptColumn when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("promptColumn"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + // promptColumn hides when task is in imageOnlyTasks or audioInputTasks; + // with undefined task, those conditions are false, so it should NOT hide + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(false); + }); + + it("should hide systemPrompt when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("systemPrompt"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); + + it("should hide contextColumn when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("contextColumn"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); + + it("should hide candidateLabels when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("candidateLabels"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); + + it("should hide sentencesColumn when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("sentencesColumn"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); + + it("should hide maxNewTokens when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("maxNewTokens"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); + + it("should hide temperature when task is undefined", () => { + initHfOperator("text-generation"); + const field = getHfField("temperature"); + const hideFn = (field?.expressions as Record<string, Function>)?.["hide"]; + expect(hideFn({ model: {} } as FormlyFieldConfig)).toBe(true); + }); }); diff --git a/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.ts b/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.ts index 2512fecdac..1c5fdbff88 100644 --- a/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.ts +++ b/frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.ts @@ -61,7 +61,7 @@ import * as Y from "yjs"; import { OperatorSchema } from "src/app/workspace/types/operator-schema.interface"; import { AttributeType, PortSchema } from "../../../types/workflow-compiling.interface"; import { GuiConfigService } from "../../../../common/service/gui-config.service"; -import { NgIf } from "@angular/common"; +import { NgFor, NgIf, NgSwitch, NgSwitchCase } from "@angular/common"; import { NzSpaceCompactItemDirective } from "ng-zorro-antd/space"; import { NzButtonComponent } from "ng-zorro-antd/button"; import { ɵNzTransitionPatchDirective } from "ng-zorro-antd/core/transition-patch"; @@ -100,6 +100,9 @@ Quill.register("modules/cursors", QuillCursors); styleUrls: ["./operator-property-edit-frame.component.scss"], imports: [ NgIf, + NgFor, + NgSwitch, + NgSwitchCase, NzSpaceCompactItemDirective, NzButtonComponent, ɵNzTransitionPatchDirective, @@ -167,6 +170,273 @@ export class OperatorPropertyEditFrameComponent implements OnInit, OnChanges, On // used to tear down subscriptions that takeUntil(teardownObservable) private teardownObservable: Subject<void> = new Subject(); + readonly huggingFaceTaskPreviewSamples: Record< + string, + { + kind: "image" | "video" | "audio" | "text"; + inputLabel?: string; + outputLabel?: string; + title?: string; + body?: string; + outputBody?: string; + pills?: string[]; + assetSrc?: string; + } + > = { + "text-to-image": { + kind: "image", + inputLabel: "Text prompt", + outputLabel: "Generated image", + title: "Comic-style city action scene", + body: "Prompt becomes a generated image preview.", + assetSrc: "assets/sample-image.png", + }, + "image-to-image": { + kind: "image", + inputLabel: "Source image", + outputLabel: "Edited image", + title: "Image transformation preview", + body: "Image input produces a modified image result.", + assetSrc: "assets/sample-image.png", + }, + "text-to-video": { + kind: "video", + inputLabel: "Text prompt", + outputLabel: "Generated video", + title: "Prompt-based motion preview", + body: "Prompt becomes a generated video clip.", + assetSrc: "assets/sample-video.mp4", + }, + "image-to-video": { + kind: "video", + inputLabel: "Source image", + outputLabel: "Animated clip", + title: "Image animation preview", + body: "Image input becomes a short generated video.", + assetSrc: "assets/sample-video.mp4", + }, + "text-to-speech": { + kind: "audio", + inputLabel: "Text input", + outputLabel: "Spoken audio", + title: "Speech synthesis preview", + body: "Text becomes an audio clip the user can play back.", + assetSrc: "assets/sample-audio.wav", + }, + "automatic-speech-recognition": { + kind: "audio", + inputLabel: "Audio input", + outputLabel: "Transcript text", + title: "Speech-to-text preview", + body: "Uploaded audio is transcribed into plain text.", + assetSrc: "assets/sample-audio.wav", + }, + "audio-classification": { + kind: "audio", + inputLabel: "Audio input", + outputLabel: "Labels and scores", + title: "Audio tagging preview", + body: "Uploaded audio returns classification labels.", + assetSrc: "assets/sample-audio.wav", + }, + "image-text-to-text": { + kind: "image", + inputLabel: "Image + text prompt", + outputLabel: "Generated text", + title: "Image-text-to-text preview", + body: "The model reads an image and a text prompt to produce a response.", + outputBody: "The image shows a superhero leaping across rooftops at sunset.", + assetSrc: "assets/sample-image.png", + }, + "image-classification": { + kind: "image", + inputLabel: "Image input", + outputLabel: "Predicted labels", + title: "Image classification preview", + body: "The model assigns labels such as superhero, city, or action scene.", + assetSrc: "assets/sample-image.png", + pills: ["superhero", "cityscape", "action"], + }, + "object-detection": { + kind: "image", + inputLabel: "Image input", + outputLabel: "Detected objects", + title: "Object detection preview", + body: "The model returns detected objects and bounding boxes.", + assetSrc: "assets/sample-image.png", + pills: ["person", "building", "sky"], + }, + "image-segmentation": { + kind: "image", + inputLabel: "Image input", + outputLabel: "Segmented regions", + title: "Segmentation preview", + body: "The model separates the image into labeled regions.", + assetSrc: "assets/sample-image.png", + pills: ["foreground", "background", "subject"], + }, + "image-to-text": { + kind: "image", + inputLabel: "Image input", + outputLabel: "Caption text", + title: "Captioning preview", + body: "The model turns an uploaded image into a textual description.", + outputBody: "A superhero leaps above a dense downtown skyline at sunset.", + assetSrc: "assets/sample-image.png", + }, + "visual-question-answering": { + kind: "image", + inputLabel: "Image + question", + outputLabel: "Answer text", + title: "Visual question answering preview", + body: "The model reads the image and answers the user question.", + outputBody: "Spider-Man is jumping over a city skyline.", + assetSrc: "assets/sample-image.png", + }, + "document-question-answering": { + kind: "image", + inputLabel: "Document image + question", + outputLabel: "Answer text", + title: "Document QA preview", + body: "The model extracts answers from a document image.", + outputBody: "Invoice total: $248.90", + assetSrc: "assets/sample-image.png", + }, + "zero-shot-image-classification": { + kind: "image", + inputLabel: "Image + candidate labels", + outputLabel: "Ranked labels", + title: "Zero-shot image labeling preview", + body: "Candidate labels are scored against the uploaded image.", + assetSrc: "assets/sample-image.png", + pills: ["superhero", "sports", "travel"], + }, + "text-generation": { + kind: "text", + inputLabel: "Prompt", + outputLabel: "Generated text", + title: "Text generation preview", + body: "Write a short action scene set above a crowded city skyline.", + outputBody: "The hero vaulted between rooftops as the city lights came alive below.", + }, + "text-classification": { + kind: "text", + inputLabel: "Text input", + outputLabel: "Predicted label", + title: "Text classification preview", + body: "This launch update sounds confident and customer-focused.", + pills: ["positive", "announcement"], + }, + "token-classification": { + kind: "text", + inputLabel: "Text input", + outputLabel: "Tagged spans", + title: "Token classification preview", + body: "Peter Parker visited New York yesterday.", + pills: ["Peter Parker: PERSON", "New York: LOCATION"], + }, + "question-answering": { + kind: "text", + inputLabel: "Question + context", + outputLabel: "Answer span", + title: "Question answering preview", + body: "Question: Who led the launch?\nContext: Maya led the launch while Jordan handled analytics.", + outputBody: "Maya", + }, + "table-question-answering": { + kind: "text", + inputLabel: "Question + table", + outputLabel: "Answer", + title: "Table QA preview", + body: "Question: Which month had the highest revenue?", + outputBody: "March", + }, + "zero-shot-classification": { + kind: "text", + inputLabel: "Text + candidate labels", + outputLabel: "Ranked labels", + title: "Zero-shot classification preview", + body: "We need to accelerate onboarding for enterprise customers.", + pills: ["business", "operations", "support"], + }, + translation: { + kind: "text", + inputLabel: "Source text", + outputLabel: "Translated text", + title: "Translation preview", + body: "Good morning, thanks for joining the call.", + outputBody: "Buenos dias, gracias por unirte a la llamada.", + }, + summarization: { + kind: "text", + inputLabel: "Long text", + outputLabel: "Summary", + title: "Summarization preview", + body: "A long project update is compressed into a short summary.", + outputBody: "The team shipped the release, fixed two regressions, and started the next milestone.", + }, + "feature-extraction": { + kind: "text", + inputLabel: "Text input", + outputLabel: "Embedding/vector output", + title: "Feature extraction preview", + body: "Input text is converted into a numeric representation.", + pills: ["0.12", "-0.08", "0.44", "..."], + }, + "fill-mask": { + kind: "text", + inputLabel: "Masked sentence", + outputLabel: "Top completions", + title: "Fill-mask preview", + body: "The hero saved the [MASK].", + pills: ["city", "day", "crowd"], + }, + "sentence-similarity": { + kind: "text", + inputLabel: "Source + candidate sentences", + outputLabel: "Similarity scores", + title: "Sentence similarity preview", + body: "Compare one sentence against several alternatives.", + pills: ["0.93", "0.61", "0.22"], + }, + "text-ranking": { + kind: "text", + inputLabel: "Query + candidate texts", + outputLabel: "Ranked results", + title: "Text ranking preview", + body: "Candidate passages are ordered by relevance to the query.", + pills: ["doc_2", "doc_5", "doc_1"], + }, + }; + + get huggingFaceTaskPreview(): { + kind: "image" | "video" | "audio" | "text"; + inputLabel?: string; + outputLabel?: string; + title?: string; + body?: string; + outputBody?: string; + pills?: string[]; + assetSrc?: string; + } | null { + if (!this.isHuggingFaceOperator()) { + return null; + } + const task = this.formData?.["task"]; + if (typeof task !== "string" || task.trim().length === 0) { + return null; + } + return ( + this.huggingFaceTaskPreviewSamples[task] ?? { + kind: "text", + inputLabel: "Task input", + outputLabel: "Task output", + title: this.formatTaskTitle(task), + body: "This task transforms the provided input into a model response.", + } + ); + } + constructor( private formlyJsonschema: FormlyJsonschema, private workflowActionService: WorkflowActionService, @@ -237,6 +507,20 @@ export class OperatorPropertyEditFrameComponent implements OnInit, OnChanges, On }); } + private isHuggingFaceOperator(): boolean { + if (!this.currentOperatorId) return false; + const graph = this.workflowActionService.getTexeraGraph(); + if (!graph.hasOperator(this.currentOperatorId)) return false; + return graph.getOperator(this.currentOperatorId).operatorType === "HuggingFace"; + } + + private formatTaskTitle(task: string): string { + return task + .split("-") + .map(part => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); + } + async ngOnDestroy() { // await this.checkAndSavePreset(); this.teardownObservable.complete(); @@ -541,6 +825,192 @@ export class OperatorPropertyEditFrameComponent implements OnInit, OnChanges, On mappedField.type = "inputautocomplete"; } + if (mappedField.key === "huggingFaceModel") { + mappedField.type = "huggingface"; + } + + if (mappedField.key === "modelId" && this.currentOperatorSchema?.operatorType === "HuggingFace") { + mappedField.type = "huggingface"; + } + + if (mappedField.key === "task" && this.currentOperatorSchema?.operatorType === "HuggingFace") { + mappedField.hide = true; + } + + // ── Dynamic field visibility for HuggingFace based on selected task ── + if (this.currentOperatorSchema?.operatorType === "HuggingFace" && typeof mappedField.key === "string") { + const hfKey = mappedField.key; + const imageOnlyTasks = ["image-classification", "object-detection", "image-segmentation", "image-to-text"]; + const imageInputTasks = [ + ...imageOnlyTasks, + "visual-question-answering", + "document-question-answering", + "zero-shot-image-classification", + "image-text-to-text", + "image-to-image", + "image-to-video", + ]; + const audioInputTasks = ["automatic-speech-recognition", "audio-classification"]; + const promptRequiredTasks = [ + "text-generation", + "text-classification", + "token-classification", + "question-answering", + "table-question-answering", + "zero-shot-classification", + "translation", + "summarization", + "feature-extraction", + "fill-mask", + "sentence-similarity", + "text-ranking", + "visual-question-answering", + "document-question-answering", + "zero-shot-image-classification", + ]; + const getSelectedTask = (field: FormlyFieldConfig): string | undefined => { + const fromForm = field.form?.get("task")?.value ?? field.formControl?.parent?.get("task")?.value; + if (typeof fromForm === "string" && fromForm.trim().length > 0) { + return fromForm; + } + const fromModel = field.model?.task; + if (typeof fromModel === "string" && fromModel.trim().length > 0) { + return fromModel; + } + return undefined; + }; + if (hfKey === "imageInput") { + mappedField.type = "huggingface-image-upload"; + mappedField.expressions = { + ...mappedField.expressions, + hide: (field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + return t === undefined || !imageInputTasks.includes(t); + }, + }; + mappedField.validators = { + ...mappedField.validators, + requiredImageInput: { + expression: (_control: AbstractControl, field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + if (t === undefined || !imageInputTasks.includes(t)) { + return true; + } + const inputImageCol = field.model?.inputImageColumn; + if (typeof inputImageCol === "string" && inputImageCol.trim().length > 0) { + return true; + } + const value = field.formControl?.value ?? field.model?.imageInput; + return typeof value === "string" && value.trim().length > 0; + }, + message: () => "Upload an image or select an Input Image Column for this task.", + }, + }; + } + if (hfKey === "audioInput") { + mappedField.type = "huggingface-audio-upload"; + mappedField.expressions = { + ...mappedField.expressions, + hide: (field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + return t === undefined || !audioInputTasks.includes(t); + }, + }; + mappedField.validators = { + ...mappedField.validators, + requiredAudioInput: { + expression: (_control: AbstractControl, field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + if (t === undefined || !audioInputTasks.includes(t)) { + return true; + } + const inputAudioCol = field.model?.inputAudioColumn; + if (typeof inputAudioCol === "string" && inputAudioCol.trim().length > 0) { + return true; + } + const value = field.formControl?.value ?? field.model?.audioInput; + return typeof value === "string" && value.trim().length > 0; + }, + message: () => "Upload audio or select an Input Audio Column for this task.", + }, + }; + } + if (hfKey === "inputImageColumn") { + mappedField.expressions = { + ...mappedField.expressions, + hide: (field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + return t === undefined || !imageInputTasks.includes(t); + }, + }; + } + if (hfKey === "inputAudioColumn") { + mappedField.expressions = { + ...mappedField.expressions, + hide: (field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + return t === undefined || !audioInputTasks.includes(t); + }, + }; + } + if (hfKey === "promptColumn") { + mappedField.expressions = { + ...mappedField.expressions, + hide: (field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + return t !== undefined && (imageOnlyTasks.includes(t) || audioInputTasks.includes(t)); + }, + }; + mappedField.validators = { + ...mappedField.validators, + requiredPromptColumn: { + expression: (_control: AbstractControl, field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + if (t === undefined || !promptRequiredTasks.includes(t)) { + return true; + } + const value = field.formControl?.value ?? field.model?.promptColumn; + return typeof value === "string" && value.trim().length > 0; + }, + message: () => "Select a prompt column for this task.", + }, + }; + } + if (["systemPrompt", "maxNewTokens", "temperature"].includes(hfKey)) { + mappedField.expressions = { + ...mappedField.expressions, + hide: (field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + return t !== "text-generation"; + }, + }; + } + if (hfKey === "contextColumn") { + mappedField.expressions = { + ...mappedField.expressions, + hide: (field: FormlyFieldConfig) => getSelectedTask(field) !== "question-answering", + }; + } + if (hfKey === "candidateLabels") { + mappedField.expressions = { + ...mappedField.expressions, + hide: (field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + return t !== "zero-shot-classification" && t !== "zero-shot-image-classification"; + }, + }; + } + if (hfKey === "sentencesColumn") { + mappedField.expressions = { + ...mappedField.expressions, + hide: (field: FormlyFieldConfig) => { + const t = getSelectedTask(field); + return t !== "sentence-similarity" && t !== "text-ranking"; + }, + }; + } + } + if (mappedField.key === "datasetVersionPath") { mappedField.type = "datasetversionselector"; } diff --git a/frontend/src/app/workspace/service/operator-metadata/mock-operator-metadata.data.ts b/frontend/src/app/workspace/service/operator-metadata/mock-operator-metadata.data.ts index b90d578e35..c0005a8c2b 100644 --- a/frontend/src/app/workspace/service/operator-metadata/mock-operator-metadata.data.ts +++ b/frontend/src/app/workspace/service/operator-metadata/mock-operator-metadata.data.ts @@ -302,6 +302,36 @@ export const mockJavaUDFSchema: OperatorSchema = { operatorVersion: "p1", }; +export const mockHuggingFaceSchema: OperatorSchema = { + operatorType: "HuggingFace", + additionalMetadata: { + userFriendlyName: "HuggingFace Inference", + operatorDescription: "Run inference on a HuggingFace model", + operatorGroupName: "Analysis", + inputPorts: [{}], + outputPorts: [{}], + }, + jsonSchema: { + properties: { + task: { type: "string", title: "task" }, + modelId: { type: "string", title: "model ID" }, + promptColumn: { type: "string", title: "prompt column" }, + imageInput: { type: "string", title: "image input" }, + audioInput: { type: "string", title: "audio input" }, + inputImageColumn: { type: "string", title: "input image column" }, + inputAudioColumn: { type: "string", title: "input audio column" }, + systemPrompt: { type: "string", title: "system prompt" }, + maxNewTokens: { type: "integer", title: "max new tokens" }, + temperature: { type: "number", title: "temperature" }, + contextColumn: { type: "string", title: "context column" }, + candidateLabels: { type: "string", title: "candidate labels" }, + sentencesColumn: { type: "string", title: "sentences column" }, + }, + type: "object", + }, + operatorVersion: "hf1", +}; + export const mockOperatorSchemaList: ReadonlyArray<OperatorSchema> = [ mockScanSourceSchema, mockFileSourceSchema, @@ -315,6 +345,7 @@ export const mockOperatorSchemaList: ReadonlyArray<OperatorSchema> = [ mockUnionSchema, mockPythonUDFSchema, mockJavaUDFSchema, + mockHuggingFaceSchema, ]; export const mockOperatorGroup: ReadonlyArray<GroupInfo> = [ diff --git a/frontend/src/app/workspace/service/workflow-graph/model/mock-workflow-data.ts b/frontend/src/app/workspace/service/workflow-graph/model/mock-workflow-data.ts index 48195c6564..a44d1fddae 100644 --- a/frontend/src/app/workspace/service/workflow-graph/model/mock-workflow-data.ts +++ b/frontend/src/app/workspace/service/workflow-graph/model/mock-workflow-data.ts @@ -122,6 +122,17 @@ export const mockPythonUDFPredicate: OperatorPredicate = { isDisabled: false, }; +export const mockHuggingFacePredicate: OperatorPredicate = { + operatorID: "8", + operatorType: "HuggingFace", + operatorVersion: "hf1", + operatorProperties: { task: "text-generation", modelId: "" }, + inputPorts: [{ portID: "input-0" }], + outputPorts: [{ portID: "output-0" }], + showAdvanced: true, + isDisabled: false, +}; + export const mockScanResultLink: OperatorLink = { linkID: "link-1", source: { diff --git a/frontend/src/assets/sample-audio.wav b/frontend/src/assets/sample-audio.wav new file mode 100644 index 0000000000..e88bea4c93 Binary files /dev/null and b/frontend/src/assets/sample-audio.wav differ diff --git a/frontend/src/assets/sample-video.mp4 b/frontend/src/assets/sample-video.mp4 new file mode 100644 index 0000000000..6b2d7fd7d7 Binary files /dev/null and b/frontend/src/assets/sample-video.mp4 differ
