s3 widget #1481

gugu · 2025-12-22T10:04:38Z

No description provided.

Copilot

Pull request overview

This PR adds an S3 widget that enables users to upload and manage files stored in AWS S3 buckets directly from table fields. The implementation spans frontend (Angular), backend (NestJS), and shared code layers.

Key Changes:

Introduces S3 widget type with AWS credential management through user secrets
Implements pre-signed URL generation for secure file uploads and downloads
Adds frontend component with file preview capabilities for images and generic files

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
`shared-code/src/shared/enums/table-widget-type.enum.ts`	Adds S3 widget type to shared enum
`backend/src/enums/widget-type.enum.ts`	Adds S3 widget type to backend enum
`backend/src/entities/widget/utils/validate-create-widgets-ds.ts`	Adds validation for S3 widget required parameters (bucket, AWS credentials)
`backend/src/entities/s3-widget/s3-helper.service.ts`	Implements S3 client creation, pre-signed URL generation, and file key generation
`backend/src/entities/s3-widget/use-cases/get-s3-file-url.use.case.ts`	Implements use case for generating pre-signed download URLs
`backend/src/entities/s3-widget/use-cases/get-s3-upload-url.use.case.ts`	Implements use case for generating pre-signed upload URLs
`backend/src/entities/s3-widget/s3-widget.controller.ts`	Exposes REST endpoints for S3 file operations with proper guards
`backend/src/entities/s3-widget/s3-widget.module.ts`	Registers S3 widget module with dependency injection and middleware
`backend/src/entities/s3-widget/use-cases/s3-use-cases.interface.ts`	Defines interfaces for S3 use cases
`backend/src/entities/s3-widget/application/data-structures/s3-widget-params.ds.ts`	Defines widget parameters data structure
`backend/src/entities/s3-widget/application/data-structures/s3-operation.ds.ts`	Defines request and response data structures for S3 operations
`backend/src/common/data-injection.tokens.ts`	Adds dependency injection tokens for S3 use cases
`backend/src/app.module.ts`	Imports S3WidgetModule into main application
`backend/package.json`	Adds AWS SDK dependencies for S3 operations
`frontend/src/app/services/s3.service.ts`	Implements Angular service for S3 API communication
`frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.ts`	Implements S3 edit component with upload and preview functionality
`frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.html`	Provides template with file input, upload button, and preview display
`frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.css`	Styles the S3 widget UI components
`frontend/src/app/consts/record-edit-types.ts`	Registers S3EditComponent in UI widget mapping
`frontend/src/app/components/dashboard/db-table-view/db-table-widgets/widget/widget.component.ts`	Adds documentation link for S3 widget
`frontend/src/app/components/dashboard/db-table-view/db-table-widgets/db-table-widgets.component.ts`	Adds configuration template and comments for S3 widget setup

Comments suppressed due to low confidence (1)

backend/src/entities/s3-widget/use-cases/get-s3-file-url.use.case.ts:1

Unused imports BadRequestException, NotFoundException.

import { HttpStatus, Inject, Injectable, NotFoundException, BadRequestException } from '@nestjs/common';

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-22T10:09:54Z

frontend/src/app/services/s3.service.ts

+import { HttpClient } from '@angular/common/http';
+import { Injectable } from '@angular/core';
+import { Observable, catchError, EMPTY } from 'rxjs';
+import { NotificationsService } from './notifications.service';
+import { AlertActionType, AlertType } from '../models/alert';
+
+interface S3FileUrlResponse {
+  url: string;
+  key: string;
+  expiresIn: number;
+}
+
+interface S3UploadUrlResponse {
+  uploadUrl: string;
+  key: string;
+  expiresIn: number;
+}
+
+@Injectable({
+  providedIn: 'root'
+})
+export class S3Service {
+  constructor(
+    private http: HttpClient,
+    private notifications: NotificationsService
+  ) {}
+
+  getFileUrl(
+    connectionId: string,
+    tableName: string,
+    fieldName: string,
+    fileKey: string
+  ): Observable<S3FileUrlResponse> {
+    return this.http.get<S3FileUrlResponse>(`/s3/file/${connectionId}`, {
+      params: { tableName, fieldName, fileKey }
+    }).pipe(
+      catchError(err => {
+        this.notifications.showAlert(
+          AlertType.Error,
+          { abstract: 'Failed to get S3 file URL', details: err.error?.message },
+          [{ type: AlertActionType.Button, caption: 'Dismiss', action: () => this.notifications.dismissAlert() }]
+        );
+        return EMPTY;
+      })
+    );
+  }
+
+  getUploadUrl(
+    connectionId: string,
+    tableName: string,
+    fieldName: string,
+    filename: string,
+    contentType: string
+  ): Observable<S3UploadUrlResponse> {
+    return this.http.post<S3UploadUrlResponse>(
+      `/s3/upload-url/${connectionId}`,
+      { filename, contentType },
+      { params: { tableName, fieldName } }
+    ).pipe(
+      catchError(err => {
+        this.notifications.showAlert(
+          AlertType.Error,
+          { abstract: 'Failed to get upload URL', details: err.error?.message },
+          [{ type: AlertActionType.Button, caption: 'Dismiss', action: () => this.notifications.dismissAlert() }]
+        );
+        return EMPTY;
+      })
+    );
+  }
+
+  uploadToS3(uploadUrl: string, file: File): Observable<void> {
+    return this.http.put<void>(uploadUrl, file, {
+      headers: { 'Content-Type': file.type }
+    }).pipe(
+      catchError(err => {
+        this.notifications.showAlert(
+          AlertType.Error,
+          { abstract: 'File upload failed', details: err.message },
+          [{ type: AlertActionType.Button, caption: 'Dismiss', action: () => this.notifications.dismissAlert() }]
+        );
+        return EMPTY;
+      })
+    );
+  }
+}


New S3Service was created but no corresponding test file (s3.service.spec.ts) was added. Based on the pattern in this codebase where services have comprehensive test coverage (e.g., auth.service.spec.ts, connections.service.spec.ts, tables.service.spec.ts), tests should be added for the new S3Service to maintain consistency.

Copilot · 2025-12-22T10:09:55Z

frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.ts

+import { Component, Input, OnInit } from '@angular/core';
+import { CommonModule } from '@angular/common';
+import { FormsModule } from '@angular/forms';
+import { MatFormFieldModule } from '@angular/material/form-field';
+import { MatInputModule } from '@angular/material/input';
+import { MatButtonModule } from '@angular/material/button';
+import { MatIconModule } from '@angular/material/icon';
+import { MatProgressSpinnerModule } from '@angular/material/progress-spinner';
+import { BaseEditFieldComponent } from '../base-row-field/base-row-field.component';
+import { S3Service } from 'src/app/services/s3.service';
+import { ConnectionsService } from 'src/app/services/connections.service';
+import { TablesService } from 'src/app/services/tables.service';
+
+interface S3WidgetParams {
+  bucket: string;
+  prefix?: string;
+  region?: string;
+  aws_access_key_id_secret_name: string;
+  aws_secret_access_key_secret_name: string;
+}
+
+@Component({
+  selector: 'app-edit-s3',
+  templateUrl: './s3.component.html',
+  styleUrl: './s3.component.css',
+  imports: [
+    CommonModule,
+    FormsModule,
+    MatFormFieldModule,
+    MatInputModule,
+    MatButtonModule,
+    MatIconModule,
+    MatProgressSpinnerModule,
+  ]
+})
+export class S3EditComponent extends BaseEditFieldComponent implements OnInit {
+  @Input() value: string;
+
+  public params: S3WidgetParams;
+  public previewUrl: string | null = null;
+  public isImage: boolean = false;
+  public isLoading: boolean = false;
+
+  private connectionId: string;
+  private tableName: string;
+
+  constructor(
+    private s3Service: S3Service,
+    private connectionsService: ConnectionsService,
+    private tablesService: TablesService
+  ) {
+    super();
+  }
+
+  ngOnInit(): void {
+    super.ngOnInit();
+    this.connectionId = this.connectionsService.currentConnectionID;
+    this.tableName = this.tablesService.currentTableName;
+    this._parseWidgetParams();
+    if (this.value) {
+      this._loadPreview();
+    }
+  }
+
+  ngOnChanges(): void {
+    this._parseWidgetParams();
+    if (this.value && !this.previewUrl && !this.isLoading) {
+      this._loadPreview();
+    }
+  }
+
+  onFileSelected(event: Event): void {
+    const input = event.target as HTMLInputElement;
+    if (!input.files?.length) return;
+
+    const file = input.files[0];
+    this.isLoading = true;
+
+    this.s3Service.getUploadUrl(
+      this.connectionId,
+      this.tableName,
+      this.widgetStructure.field_name,
+      file.name,
+      file.type
+    ).subscribe({
+      next: (response) => {
+        this.s3Service.uploadToS3(response.uploadUrl, file).subscribe({
+          next: () => {
+            this.value = response.key;
+            this.onFieldChange.emit(response.key);
+            this._loadPreview();
+          },
+          error: () => {
+            this.isLoading = false;
+          }
+        });
+      },
+      error: () => {
+        this.isLoading = false;
+      }
+    });
+  }
+
+  openFile(): void {
+    if (this.previewUrl) {
+      window.open(this.previewUrl, '_blank');
+    }
+  }
+
+  private _parseWidgetParams(): void {
+    if (this.widgetStructure?.widget_params) {
+      try {
+        this.params = typeof this.widgetStructure.widget_params === 'string'
+          ? JSON.parse(this.widgetStructure.widget_params)
+          : this.widgetStructure.widget_params;
+      } catch (e) {
+        console.error('Error parsing S3 widget params:', e);
+      }
+    }
+  }
+
+  private _loadPreview(): void {
+    if (!this.value || !this.connectionId || !this.tableName) return;
+
+    this.isLoading = true;
+    this.isImage = this._isImageFile(this.value);
+
+    this.s3Service.getFileUrl(
+      this.connectionId,
+      this.tableName,
+      this.widgetStructure.field_name,
+      this.value
+    ).subscribe({
+      next: (response) => {
+        this.previewUrl = response.url;
+        this.isLoading = false;
+      },
+      error: () => {
+        this.isLoading = false;
+      }
+    });
+  }
+
+  private _isImageFile(key: string): boolean {
+    const imageExtensions = ['.jpg', '.jpeg', '.png', '.gif', '.webp', '.svg', '.bmp'];
+    const lowerKey = key.toLowerCase();
+    return imageExtensions.some(ext => lowerKey.endsWith(ext));
+  }
+}


The new S3EditComponent was added but no corresponding test file (s3.component.spec.ts) exists. Following the pattern established in this codebase where similar edit components have test files (e.g., file.component.spec.ts exists for FileEditComponent), tests should be added for the S3EditComponent to maintain test coverage consistency.

Copilot · 2025-12-22T10:09:55Z

frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.html

+  </mat-form-field>
+
+  <div class="s3-widget__actions">
+    <input type="file" #fileInput hidden (change)="onFileSelected($event)">


The file input element doesn't have an 'accept' attribute to restrict file types. Without this, users can select any file type, which could lead to confusion or unnecessary uploads. Consider adding an accept attribute (e.g., accept="/" or specific MIME types) to improve user experience and clarify expected file types.

Suggested change

<input type="file" #fileInput hidden (change)="onFileSelected($event)">

<input type="file" #fileInput hidden accept="*/*" (change)="onFileSelected($event)">

Copilot · 2025-12-22T10:09:55Z

frontend/src/app/services/s3.service.ts

+      catchError(err => {
+        this.notifications.showAlert(
+          AlertType.Error,
+          { abstract: 'File upload failed', details: err.message },


The uploadToS3 method performs an HTTP PUT directly to the pre-signed S3 URL. However, in case of a network error or S3 service error, the error object may not have a standard 'message' property (line 78), unlike the other error handlers which correctly access err.error?.message. This inconsistency means users may see undefined error details. Consider changing to access err.error?.message or provide a fallback error message.

Suggested change

{ abstract: 'File upload failed', details: err.message },

{ abstract: 'File upload failed', details: err.error?.message || err.message || 'An unexpected error occurred during file upload.' },

Copilot · 2025-12-22T10:09:56Z

backend/src/entities/s3-widget/use-cases/get-s3-file-url.use.case.ts

@@ -0,0 +1,83 @@
+import { HttpStatus, Inject, Injectable, NotFoundException, BadRequestException } from '@nestjs/common';


Unused imports NotFoundException and BadRequestException are imported but never used in the file. HttpException is used instead for all error cases. Remove these unused imports to keep the code clean.

Suggested change

import { HttpStatus, Inject, Injectable, NotFoundException, BadRequestException } from '@nestjs/common';

import { HttpStatus, Inject, Injectable } from '@nestjs/common';

Copilot · 2025-12-22T10:09:56Z

frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.ts

+            this.isLoading = false;
+          }
+        });
+      },
+      error: () => {
+        this.isLoading = false;


When uploading a file, if getUploadUrl fails, isLoading is reset to false. However, if getUploadUrl succeeds but uploadToS3 fails, isLoading is also reset. The issue is that in both error scenarios, the preview is not reloaded for the old value. If the user had a previous file loaded with a preview, and the upload fails, the previewUrl remains null even though the value field still contains the old key. Consider reloading the preview on upload failure if there's still a value.

Suggested change

this.isLoading = false;

}

});

},

error: () => {

this.isLoading = false;

this.isLoading = false;

if (this.value) {

this._loadPreview();

}

}

});

},

error: () => {

this.isLoading = false;

if (this.value) {

this._loadPreview();

}

Copilot · 2025-12-22T10:09:56Z

frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.ts

@@ -0,0 +1,149 @@
+import { Component, Input, OnInit } from '@angular/core';


The component implements ngOnChanges lifecycle hook but does not import OnChanges from @angular/core. Add OnChanges to the import statement to match the implementation pattern used by other components like ImageEditComponent and UrlEditComponent.

Suggested change

import { Component, Input, OnInit } from '@angular/core';

import { Component, Input, OnInit, OnChanges } from '@angular/core';

Copilot · 2025-12-22T10:09:57Z

backend/src/entities/s3-widget/s3-helper.service.ts

+    if (prefix) {
+      const normalizedPrefix = prefix.replace(/\/$/, '');
+      return `${normalizedPrefix}/${timestamp}_${sanitizedFilename}`;
+    }
+    return `${timestamp}_${sanitizedFilename}`;


The file key sanitization pattern replaces non-alphanumeric characters (except dot, underscore, and hyphen) with underscores. However, this doesn't handle potentially problematic patterns like multiple consecutive dots (e.g., "..") or leading dots which could create hidden files or path traversal issues. Consider adding validation to reject filenames with these patterns or further sanitize them.

Suggested change

if (prefix) {

const normalizedPrefix = prefix.replace(/\/$/, '');

return `${normalizedPrefix}/${timestamp}_${sanitizedFilename}`;

}

return `${timestamp}_${sanitizedFilename}`;

// Further sanitize to avoid problematic patterns like leading dots or multiple consecutive dots

let safeFilename = sanitizedFilename

// Remove any leading dots to avoid hidden-file style names

.replace(/^\.+/, '')

// Replace multiple consecutive dots with a single underscore

.replace(/\.\.+/g, '_');

// Ensure we still have a usable filename

if (!safeFilename) {

safeFilename = 'file';

}

if (prefix) {

const normalizedPrefix = prefix.replace(/\/$/, '');

return `${normalizedPrefix}/${timestamp}_${safeFilename}`;

}

return `${timestamp}_${safeFilename}`;

Copilot · 2025-12-22T10:09:57Z

frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.html

+         alt="Preview">
+    <div *ngIf="!isImage && previewUrl && !isLoading" class="s3-widget__file-icon">
+      <mat-icon>insert_drive_file</mat-icon>
+      <span class="s3-widget__filename">{{value | slice:-30}}</span>


Using slice:-30 to display the last 30 characters of the file key may show only part of the filename or path components, making it difficult for users to identify files. For example, with a timestamp prefix like "1234567890_filename.pdf", users might only see "0_filename.pdf". Consider showing the actual filename portion (after the timestamp prefix) or the first N characters instead of the last N.

Suggested change

<span class="s3-widget__filename">{{value | slice:-30}}</span>

<span class="s3-widget__filename">{{value}}</span>

s3 widget

9e3a7d8

gugu requested review from Artuomka, Copilot and lyubov-voloshko December 22, 2025 10:04

Copilot started reviewing on behalf of gugu December 22, 2025 10:05 View session

Copilot AI reviewed Dec 22, 2025

View reviewed changes

gugu added 6 commits December 22, 2025 10:55

s3 widget by row pk instead of filename

76c5dc0

tests fix

7e87bcc

Merge branch 'main' into s3-widget

81916b2

frontend tests

07004eb

Merge branch 'main' into s3-widget

ae10647

missing dependency

c08f88c

gugu enabled auto-merge (squash) December 24, 2025 14:47

gugu merged commit 56e7c4e into main Dec 24, 2025
19 of 25 checks passed

gugu deleted the s3-widget branch December 24, 2025 14:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

s3 widget #1481

s3 widget #1481

Uh oh!

gugu commented Dec 22, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Copilot AI Dec 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	<input type="file" #fileInput hidden (change)="onFileSelected($event)">
	<input type="file" #fileInput hidden accept="/" (change)="onFileSelected($event)">

	{ abstract: 'File upload failed', details: err.message },
	{ abstract: 'File upload failed', details: err.error?.message \|\| err.message \|\| 'An unexpected error occurred during file upload.' },

		@@ -0,0 +1,83 @@
		import { HttpStatus, Inject, Injectable, NotFoundException, BadRequestException } from '@nestjs/common';

	import { HttpStatus, Inject, Injectable, NotFoundException, BadRequestException } from '@nestjs/common';
	import { HttpStatus, Inject, Injectable } from '@nestjs/common';

		@@ -0,0 +1,149 @@
		import { Component, Input, OnInit } from '@angular/core';

	import { Component, Input, OnInit } from '@angular/core';
	import { Component, Input, OnInit, OnChanges } from '@angular/core';

-    if (prefix) {
-      const normalizedPrefix = prefix.replace(/\/$/, '');
-      return `${normalizedPrefix}/${timestamp}_${sanitizedFilename}`;
-    }
-    return `${timestamp}_${sanitizedFilename}`;
+    // Further sanitize to avoid problematic patterns like leading dots or multiple consecutive dots
+    let safeFilename = sanitizedFilename
+      // Remove any leading dots to avoid hidden-file style names
+      .replace(/^\.+/, '')
+      // Replace multiple consecutive dots with a single underscore
+      .replace(/\.\.+/g, '_');
+    // Ensure we still have a usable filename
+    if (!safeFilename) {
+      safeFilename = 'file';
+    }
+    if (prefix) {
+      const normalizedPrefix = prefix.replace(/\/$/, '');
+      return `${normalizedPrefix}/${timestamp}_${safeFilename}`;
+    }
+    return `${timestamp}_${safeFilename}`;

	<span class="s3-widget__filename">{{value \| slice:-30}}</span>
	<span class="s3-widget__filename">{{value}}</span>

Uh oh!

s3 widget #1481

s3 widget #1481

Uh oh!

Conversation

gugu commented Dec 22, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants