Initial commit: file_manager package with local and S3 support

This commit is contained in:
loveuer
2026-01-17 15:19:50 +08:00
commit f7160ce416
10 changed files with 1526 additions and 0 deletions

101
README.md Normal file
View File

@@ -0,0 +1,101 @@
# upkg
Go utility packages collection.
## file_manager
File management abstraction layer supporting multiple storage backends.
### Features
- **Local Storage**: Store files on local filesystem with manifest-based metadata
- **S3 Compatible Storage**: Support AWS S3, MinIO, and other S3-compatible services
- **SHA256 Verification**: Optional file integrity verification
- **Automatic Cleanup**: Configurable timeout for incomplete uploads and expiration for completed files
- **Persistent State**: State managed via manifest files (local) or object metadata (S3)
### Usage
```go
import "gitea.loveuer.com/loveuer/upkg/controller/file_manager"
```
#### Local Storage
```go
fm := file_manager.New(
file_manager.WithDir("/path/to/uploads"),
file_manager.WithTimeout(5*time.Minute),
file_manager.WithExpire(24*time.Hour),
)
defer fm.CloseManager()
```
#### S3 Storage
```go
fm := file_manager.New(
file_manager.WithS3(
"http://minio:9000",
"access-key",
"secret-key",
"bucket-name",
"us-east-1",
),
file_manager.WithS3PathStyle(true),
file_manager.WithTimeout(10*time.Minute),
file_manager.WithExpire(48*time.Hour),
)
defer fm.CloseManager()
```
#### API
```go
// Create creates a file record, returns code for upload
result, err := fm.Create(ctx, "filename.txt", 1024, "")
// Upload uploads file content
total, written, err := fm.Upload(ctx, code, 0, 1024, reader)
// Get retrieves file content
data, err := fm.Get(ctx, code)
// GetInfo returns file metadata
info, err := fm.GetInfo(ctx, code)
// Delete removes file
err := fm.Delete(ctx, code)
// Close closes file handle
err := fm.Close(code)
// CloseManager shuts down the manager
fm.CloseManager()
```
### Manifest Format (Local)
```json
{
"filename": "file.txt",
"size": 1024,
"sha256": "abc123...",
"path": "/uploads/abc123",
"create_time": "2024-01-01T00:00:00Z",
"complete": true
}
```
### S3 Object Metadata
- `x-amz-meta-filename`: Original filename
- `x-amz-meta-size`: File size
- `x-amz-meta-sha256`: SHA256 hash (optional)
- `x-amz-meta-create-time`: Creation timestamp
- `x-amz-meta-complete`: Upload completion status
- `x-amz-meta-code`: Unique file code
## License
MIT