❤
npm
Sign UpSign In

uniqhtt
TypeScript icon, indicating that this package has built-in type declarations

1.2.6 • Public • Published

Uniqhtt

A sophisticated, enterprise-grade HTTP client for Node.js and edge environments featuring intelligent cookie management, advanced web crawling capabilities, and comprehensive automation tools.

npm version TypeScript License: MIT

Table of Contents

Overview

Uniqhtt represents a paradigm shift in HTTP client architecture, engineered specifically for modern applications requiring sophisticated request management, intelligent session handling, and seamless web automation. Built with a dual-runtime foundation, it delivers consistent performance across Node.js environments and edge computing platforms while maintaining enterprise-grade reliability and security standards.

Key Architectural Highlights

  • 🌐 Universal Runtime Compatibility - Seamlessly operates across Node.js, Deno, Bun, and edge environments including Cloudflare Workers
  • 🍪 Intelligent Cookie Ecosystem - Advanced session management powered by tough-cookie with automatic persistence, domain handling, and multi-format serialization
  • 🕷️ Enterprise Web Crawler - Production-ready crawling engine featuring DOM manipulation, intelligent caching, and event-driven architecture
  • 🔄 Adaptive Request Orchestration - Sophisticated queue management with configurable concurrency, rate limiting, and priority handling
  • 🛡️ Multi-Protocol Proxy Integration - Comprehensive proxy support including SOCKS4/5, HTTP/HTTPS with authentication and failover mechanisms
  • 📥 High-Performance File Operations - Memory-efficient streaming downloads with real-time progress tracking and metadata extraction
  • 🔄 Resilient Retry Framework - Intelligent retry logic with exponential backoff, circuit breaking, and conditional retry strategies
  • 🗜️ Transparent Content Processing - Automatic decompression handling for gzip, brotli, and deflate with streaming optimization
  • 🔐 Security-First Design - Modern TLS contexts, certificate validation, secure cookie handling, and header sanitization
  • 📊 TypeScript Native - Complete type safety with comprehensive interfaces and generic support for enhanced developer experience

Installation

npm install uniqhtt
yarn add uniqhtt
pnpm add uniqhtt

Quick Start

Basic HTTP Requests

Experience the elegance of Uniqhtt's intuitive API designed for both simplicity and power:

import uniqhtt from 'uniqhtt';

// Effortless GET request with automatic response parsing
const response = await uniqhtt.get<User[]>('https://api.example.com/users');
console.log(response.data); // Fully typed response data

// Streamlined JSON POST with intelligent content-type handling
const user = await uniqhtt.postJson<User>('https://api.example.com/users', {
  name: 'John Doe',
  email: 'john@example.com',
  preferences: { theme: 'dark', notifications: true }
});

// Form-encoded POST with automatic serialization
const loginResult = await uniqhtt.postForm<AuthResponse>('https://api.example.com/login', {
  username: 'john',
  password: 'secret',
  rememberMe: true
});

console.log(`Welcome ${loginResult.data.user.name}!`);

File Downloads

Uniqhtt's file download capabilities provide enterprise-grade performance with comprehensive progress tracking and metadata extraction:

import { DownloadResponse } from 'uniqhtt';

// Direct-to-disk streaming download with automatic directory creation
const downloadResult: DownloadResponse<null> = await uniqhtt.download(
  'https://example.com/large-dataset.zip',
  './downloads/dataset.zip'
);

const downloadResult2: DownloadResponse<null> = await uniqhtt.post('https://example.com/large-dataset.zip', {},
        {saveTo: './downloads/dataset.zip'}
);

console.log(`✅ Download completed successfully:`);
console.log(`   File: ${downloadResult.fileName}`);
console.log(`   Size: ${downloadResult.size}`);
console.log(`   Speed: ${downloadResult.downloadSpeed}`);
console.log(`   Duration: ${downloadResult.totalTime}`);

// Alternative syntax for file downloads with full response access
const fileResponse = await uniqhtt.get<null>('https://example.com/report.pdf', {
  saveTo: './reports/monthly-report.pdf'
});

Core Features

Cookie Management

Uniqhtt's cookie management system represents a sophisticated approach to session handling, providing seamless persistence and intelligent domain management across complex web interactions:

import { Uniqhtt, Cookie, SerializedCookie } from 'uniqhtt';

const client = new Uniqhtt();

// Programmatic cookie creation with comprehensive attribute support
const sessionCookie = new Cookie({
  key: 'session_token',
  value: 'eyJhbGciOiJIUzI1NiIs...',
  domain: '.example.com',
  path: '/',
  secure: true,
  httpOnly: true,
  sameSite: 'strict',
  expires: new Date(Date.now() + 24 * 60 * 60 * 1000) // 24 hours
});

client.setCookies([sessionCookie]);

// Bulk cookie import from various formats
const netscapeCookies = `
# Netscape HTTP Cookie File
.example.com	TRUE	/	TRUE	${Date.now() + 86400000}	auth_token	abc123
.example.com	TRUE	/api	FALSE	0	api_key	xyz789
`;
client.setCookies(netscapeCookies);

// Automatic cookie persistence across requests
const dashboardData = await client.get<DashboardData>('https://example.com/dashboard');
const apiResponse = await client.post<ApiResult>('https://example.com/api/data', payload);

// Advanced cookie introspection and management
const currentCookies = client.getCookies();
console.log('Cookie String:', currentCookies.string);        // HTTP Cookie header format
console.log('Netscape Format:', currentCookies.netscape);    // For browser import/export
console.log('Serialized:', currentCookies.serialized);      // For JSON storage

Proxy Configuration

Uniqhtt's proxy infrastructure supports enterprise networking environments with comprehensive protocol support and intelligent failover mechanisms:

import { Uniqhtt, IProxy } from 'uniqhtt';

// Advanced SOCKS5 proxy configuration with authentication
const socksProxy: IProxy = {
  protocol: 'socks5',
  host: '127.0.0.1',
  port: 9050,
  username: 'proxy_user',
  password: 'secure_password',
  keepAlive: true,
  timeout: 30000,
  maxSockets: 10
};

const client = new Uniqhtt({ proxy: socksProxy });

// Corporate HTTP proxy with custom configuration
const corporateProxy: IProxy = {
  protocol: 'https',
  host: 'proxy.corporate.com',
  port: 8080,
  username: 'employee_id',
  password: 'access_token',
  rejectUnauthorized: false, // For internal certificates
  keepAliveMsecs: 60000
};

// All requests automatically route through configured proxy
const externalData = await client.get<ExternalApiResponse>('https://api.external.com/data');
console.log('Request routed through proxy:', externalData.status);

Request Queue & Rate Limiting

The integrated queue management system provides sophisticated traffic control, ensuring respectful interaction with target servers while maximizing throughput efficiency:

import { Uniqhtt, queueOptions } from 'uniqhtt';

// Enterprise-grade queue configuration for high-throughput scenarios
const queueConfig: queueOptions = {
  concurrency: 5,           // Maximum simultaneous requests
  interval: 1000,           // Time window in milliseconds  
  intervalCap: 2,           // Requests allowed per interval
  timeout: 30000,           // Individual request timeout
  throwOnTimeout: true,     // Fail fast on queue timeout
  autoStart: true           // Begin processing immediately
};

const client = new Uniqhtt({
  queueOptions: {
    enable: true,
    options: queueConfig
  }
});

// Bulk operations automatically respect queue constraints
const endpoints: string[] = [
  'https://api.example.com/users',
  'https://api.example.com/products', 
  'https://api.example.com/orders',
  // ... hundreds more URLs
];

const results = await Promise.all(
  endpoints.map((url, index) => 
    client.get<ApiResponse>(`${url}?page=${index}`)
  )
); // Automatically throttled to queue specifications

console.log(`Processed ${results.length} requests with optimal rate limiting`);

Retry Logic

Uniqhtt's retry framework implements intelligent failure recovery with adaptive backoff strategies and conditional retry logic:

import { UniqhttResponse, UniqhttError } from 'uniqhtt';

// Sophisticated retry configuration with exponential backoff
const resilientResponse = await uniqhtt.get<CriticalData>('https://unreliable-api.com/critical-data', {
  retry: {
    maxRetries: 5,
    retryDelay: 1000,        // Initial delay: 1 second
    incrementDelay: true     // Exponential backoff: 1s, 2s, 4s, 8s, 16s
  },
  timeout: 10000
});

console.log('Data retrieved successfully:', resilientResponse.data);

Multipart Form Data

Advanced multipart handling with comprehensive file upload support and intelligent content-type detection:

import { FormData, UniqhttResponse } from 'uniqhtt';
import { readFileSync } from 'fs';

// Comprehensive multipart form construction
const formData = new FormData();
formData.append('document', readFileSync('./contract.pdf'), {
  filename: 'contract-v2.pdf',
  contentType: 'application/pdf'
});
formData.append('metadata', JSON.stringify({
  title: 'Service Agreement',
  version: '2.1',
  confidential: true
}), { contentType: 'application/json' });
formData.append('signature', 'John Doe');
formData.append('timestamp', new Date().toISOString());

// Streamlined multipart upload with progress tracking
const uploadResult = await uniqhtt.postMultipart<UploadResponse>(
  'https://api.example.com/documents/upload',
  formData,
  {
    timeout: 120000, // Extended timeout for large files
    headers: {
      'X-Upload-Session': 'session_12345',
      'X-Client-Version': '1.0.0'
    }
  }
);

console.log(`Document uploaded: ${uploadResult.data.documentId}`);

Web Crawler

Uniqhtt's integrated crawler represents a production-ready web automation platform, featuring intelligent DOM manipulation, advanced caching strategies, and event-driven data extraction architecture.

Basic Crawling

The crawler provides an elegant, declarative approach to web data extraction with automatic queue management and intelligent content processing:

import uniqhtt from 'uniqhtt';

// Configure enterprise crawler with comprehensive options
const crawler = uniqhtt.crawler({
  crawlerOptions: {
    queueOptions: { concurrency: 3 },
    baseUrl: 'https://ecommerce.example.com',
    enableCache: true,
    cacheTTL: 1000 * 60 * 60, // 1 hour cache
    timeout: 15000,
    maxRetryAttempts: 3,
    retryDelay: 2000,
    headers: {
      'User-Agent': 'Uniqhtt-Crawler/1.0 (+https://company.com/bot)',
      'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
    }
  }
});

// Event-driven data extraction with intelligent selectors
crawler
  .onSelection<HTMLAnchorElement>('a[href*="/products/"]', (productLink) => {
    const productUrl = productLink.getAttribute('href');
    if (productUrl) {
      crawler.visit(productUrl); // Queues for processing
    }
  })
  .onSelection<HTMLElement>('.product-title', (titleElement) => {
    console.log('Product Found:', titleElement.textContent?.trim());
  })
  .onSelection<HTMLElement>('.price', (priceElement) => {
    const price = priceElement.textContent?.match(/\$(\d+\.?\d*)/)?.[1];
    if (price) {
      console.log('Price:', parseFloat(price));
    }
  })
  .onJsonData<ProductData>((apiData) => {
    // Handle JSON API responses within pages
    console.log('API Data:', apiData.products.length, 'products loaded');
  });

// Initiate crawling process (synchronous queue addition)
crawler.visit('/catalog');
crawler.visit('/featured-products');
crawler.visit('/new-arrivals');

// Wait for all queued operations to complete (asynchronous)
await crawler.waitForAll(); // Wait for all queued requests
console.log('✅ Crawling completed successfully');

Dual HTTP Client Architecture

Advanced crawler configuration supporting differentiated handling of internal and external requests:

// Primary crawler for main domain with proxy routing
const advancedCrawler = uniqhtt.crawler({
  crawlerOptions: {
    queueOptions: { concurrency: 4 },
    baseUrl: 'https://main-site.com',
    socksProxies: [
      {
        protocol: 'socks5',
        host: '127.0.0.1',
        port: 9050
      }
    ],
    retryOnProxyError: true,
    enableCache: true,
    debug: true
  },
  
  // Secondary client for external domain handling
  crawler2Options: {
    timeout: 8000,
    useProxy: false,           // Bypass proxy for external requests
    concurrency: 2,            // Lower concurrency for external sites
    respectQueueOption: true,  // Honor queue settings
    maxRetryAttempts: 1        // Reduced retries for external resources
  }
});

// Intelligent request routing based on domain
advancedCrawler
  .onSelection<HTMLAnchorElement>('a[href]', (link) => {
    const href = link.getAttribute('href');
    if (!href) return;
    
    const url = new URL(href, 'https://main-site.com');
    
    if (url.hostname === 'main-site.com') {
      // Internal links use primary client (with proxy)
      crawler.visit(url.pathname);
    } else {
      // External links use secondary client (direct connection)
      crawler.visit2(url.href);
    }
  });

crawler.visit('/');
await crawler.waitForAll(); // Wait for all queued requests

Event-Driven Data Extraction

Comprehensive event system for sophisticated data extraction and processing workflows:

interface ProductData {
  id: string;
  name: string;
  price: number;
  availability: boolean;
}

const dataCrawler = uniqhtt.crawler({
  crawlerOptions: {
    queueOptions: { concurrency: 5 },
    baseUrl: 'https://api-driven-site.com',
    enableCache: true
  }
});

// Multiple event handlers for comprehensive data capture
dataCrawler
  .onDocument((document) => {
    // Full document processing
    const title = document.querySelector('title')?.textContent;
    console.log('Page Title:', title);
  })
  .onResponse<string>((response) => {
    // Access to full HTTP response
    console.log(`Response: ${response.status} - ${response.finalUrl}`);
    console.log(`Content-Type: ${response.contentType}`);
  })
  .onJsonData<ProductData[]>((jsonData) => {
    // Automatic JSON parsing and type safety
    jsonData.forEach(product => {
      console.log(`Product: ${product.name} - $${product.price}`);
    });
  })
  .onRawData((buffer) => {
    // Raw response data access for custom processing
    console.log(`Received ${buffer.length} bytes`);
  })
  .onAnchor((anchor) => {
    // Specialized anchor element handling
    const href = anchor.href;
    const text = anchor.textContent;
    console.log(`Link: ${text} -> ${href}`);
  })
  .onAttribute('data-product-id', (productId) => {
    // Extract specific attributes across all elements
    console.log('Product ID found:', productId);
  });

dataCrawler.visit('/products');
await dataCrawler.waitForAll(); // Wait for all queued requests

Configuration

Global Configuration

Establish consistent behavior across all requests with comprehensive global settings:

import { Uniqhtt } from 'uniqhtt';

const client = new Uniqhtt({
  baseURL: 'https://api.enterprise.com/v2',
  timeout: 30000,
  
  headers: {
    'User-Agent': 'Enterprise-App/2.1.0',
    'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIs...',
    'X-API-Version': '2.1',
    'X-Client-ID': 'app_12345'
  },
  
  // Browser behavior simulation
  mimicBrowser: true,
  
  // Global retry policy
  retry: {
    maxRetries: 3,
    retryDelay: 1500,
    incrementDelay: true
  },
  
  // Advanced networking options
  rejectUnauthorized: true,
  useSecureContext: true,
  
  // Cookie management
  enableCookieJar: true,
  
  // Queue configuration for all requests
  queueOptions: {
    enable: true,
    options: {
      concurrency: 10,
      interval: 500,
      intervalCap: 5
    }
  }
});

Per-Request Configuration

Fine-tune individual requests with granular control over behavior and processing:

import { UniqhttResponse } from 'uniqhtt';

const customResponse = await uniqhtt.get<ApiData>('https://api.example.com/sensitive-data', {
  headers: { 
    'X-Request-ID': crypto.randomUUID(),
    'X-Priority': 'high'
  },
  
  timeout: 5000,
  maxRedirects: 3,
  
  // Custom response validation
  validateStatus: (status: number): boolean => {
    return status >= 200 && status < 300;
  },
  
  // Redirect handling with custom logic
  onRedirect: ({ url, status, sameDomain, method }) => {
    console.log(`Redirecting to: ${url.href} (${status})`);
    
    if (!sameDomain) {
      console.warn('Cross-domain redirect detected');
      return { 
        redirect: false,
        message: 'Cross-domain redirects not allowed'
      };
    }
    
    return { 
      redirect: true,
      method: status === 303 ? 'GET' : method,
      withoutBody: status === 303
    };
  },
  
  // Custom error recovery
  retry: {
    maxRetries: 2,
    retryDelay: 1000,
    incrementDelay: false
  }
});

TypeScript Support

Uniqhtt's TypeScript integration provides comprehensive type safety with intelligent inference and generic support:

import uniqhtt, { 
  UniqhttResponse, 
  UniqhttError, 
  HttpConfig,
  DownloadResponse 
} from 'uniqhtt';

// Define API response interfaces
interface User {
  id: number;
  name: string;
  email: string;
  profile: {
    avatar_url: string;
    preferences: Record<string, any>;
  };
}

interface ApiError {
  error: string;
  code: number;
  details?: string[];
}

// Fully typed request with comprehensive error handling
async function fetchUserData(userId: number): Promise<User> {
  try {
    const response: UniqhttResponse<User> = await uniqhtt.get<User>(
      `https://api.example.com/users/${userId}`,
      {
        headers: {
          'Accept': 'application/json',
          'X-Request-ID': crypto.randomUUID()
        },
        timeout: 10000
      } as HttpConfig
    );
    
    // Full type safety on response properties
    console.log(`User: ${response.data.name}`);
    console.log(`Status: ${response.status}`);
    console.log(`Content-Type: ${response.contentType}`);
    console.log(`Final URL: ${response.finalUrl}`);
    
    return response.data;
    
  } catch (error) {
    if (error instanceof UniqhttError) {
      // Typed error handling
      const errorData = error.response.data as ApiError;
      console.error(`API Error: ${errorData.error} (${errorData.code})`);
      console.error(`Status: ${error.response.status}`);
      console.error(`URL: ${error.response.finalUrl}`);
    }
    throw error;
  }
}

// Generic utility for typed API calls
async function apiCall<T, E = ApiError>(
  endpoint: string,
  options?: HttpConfig
): Promise<T> {
  const response = await uniqhtt.get<T>(endpoint, options);
  return response.data;
}

// Usage with full type inference
const users = await apiCall<User[]>('/users');
const settings = await apiCall<AppSettings>('/settings');

Environment Detection

Uniqhtt automatically adapts to different runtime environments while maintaining consistent API behavior:

// Node.js environment - Full feature set including file system operations
import uniqhtt from 'uniqhtt/nodejs';
import { UniqhttNode } from 'uniqhtt';

const nodeClient = new UniqhttNode({
  useCurl: true,  // Enable cURL backend for advanced features
  httpAgent: customHttpAgent,
  httpsAgent: customHttpsAgent
});

// Edge environment - Optimized for serverless and edge computing
import uniqhtt from 'uniqhtt/edge';
import { UniqhttEdge } from 'uniqhtt';

const edgeClient = new UniqhttEdge({
  timeout: 5000,  // Shorter timeouts for edge environments
  maxRedirects: 3
});

// Automatic detection (recommended) - Chooses optimal implementation
import uniqhtt, { Uniqhtt } from 'uniqhtt';

// Runtime detection happens automatically
const response = await uniqhtt.get('https://api.example.com/data');

Error Handling

Comprehensive error management with detailed context and intelligent recovery strategies:

import { UniqhttError, UniqhttResponse } from 'uniqhtt';

async function robustApiCall<T>(url: string): Promise<T> {
  try {
    const response = await uniqhtt.get<T>(url, {
      retry: {
        maxRetries: 3,
        retryDelay: 1000,
        incrementDelay: true
      },
      timeout: 15000
    });
    
    return response.data;
    
  } catch (error) {
    if (error instanceof UniqhttError) {
      // Comprehensive error context
      const errorInfo = {
        message: error.message,
        code: error.code,
        status: error.response.status,
        statusText: error.response.statusText,
        url: error.response.finalUrl,
        headers: error.response.headers,
        data: error.response.data
      };
      
      console.error('Request failed:', errorInfo);
      
      // Conditional error handling based on status
      switch (error.response.status) {
        case 401:
          console.error('Authentication required');
          // Trigger re-authentication flow
          break;
        case 403:
          console.error('Access forbidden');
          // Handle authorization failure
          break;
        case 429:
          console.error('Rate limit exceeded');
          // Implement backoff strategy
          break;
        case 500:
        case 502:
        case 503:
        case 504:
          console.error('Server error - retry may succeed');
          // Server errors might be temporary
          break;
        default:
          console.error('Unexpected error');
      }
    } else {
      // Handle non-Uniqhtt errors
      console.error('Unknown error:', error);
    }
    
    throw error;
  }
}

HTTP Methods Reference

Complete API reference for all supported HTTP methods with consistent TypeScript signatures:

// Core HTTP methods with full type support
const getData = await uniqhtt.get<ResponseType>(url, config);
const postResult = await uniqhtt.post<ResponseType>(url, data, config);
const putResult = await uniqhtt.put<ResponseType>(url, data, config);
const patchResult = await uniqhtt.patch<ResponseType>(url, data, config);
const deleteResult = await uniqhtt.delete<ResponseType>(url, config);
const headResult = await uniqhtt.head<ResponseType>(url, config);
const optionsResult = await uniqhtt.options<ResponseType>(url, config);

// Specialized content-type methods
const jsonResponse = await uniqhtt.postJson<ApiResponse>(url, jsonData, config);
const formResponse = await uniqhtt.postForm<FormResult>(url, formData, config);
const multipartResponse = await uniqhtt.postMultipart<UploadResult>(url, formData, config);

// File operations
const downloadResult: DownloadResponse<null> = await uniqhtt.download(url, localPath, config);

// Method variants for different HTTP verbs
const putJsonResult = await uniqhtt.putJson<UpdateResult>(url, jsonData, config);
const patchFormResult = await uniqhtt.patchForm<PatchResult>(url, formData, config);

// Buffer response handling
const bufferResponse: UniqhttResponse<Buffer> = await uniqhtt.get(url, { 
  returnBuffer: true 
});

Performance & Security

Performance Optimizations

Uniqhtt implements numerous performance enhancements for optimal throughput and resource utilization:

  • Intelligent Connection Pooling: Automatic HTTP/HTTPS agent management with configurable pool sizes and keep-alive settings
  • Streaming Architecture: Memory-efficient processing for large files and datasets with minimal memory footprint
  • Adaptive Compression: Automatic negotiation and decompression of gzip, brotli, and deflate with streaming optimization
  • Smart Caching: Built-in response caching for crawler operations with TTL management and intelligent invalidation
  • Queue Optimization: Sophisticated request queuing with priority handling and adaptive concurrency control

Security Features

Enterprise-grade security implementations ensure safe operation in production environments:

  • Modern TLS Configuration: Custom TLS contexts with contemporary cipher suites and protocol version enforcement
  • Certificate Validation: Comprehensive certificate chain validation with custom CA support
  • Secure Cookie Handling: Proper implementation of secure, httpOnly, and SameSite cookie attributes
  • Header Sanitization: Automatic cleanup and validation of security-sensitive headers
  • Proxy Security: Secure credential handling for authenticated proxy connections with encryption support
// Security-focused configuration example
const secureClient = new Uniqhtt({
  rejectUnauthorized: true,
  useSecureContext: true,
  headers: {
    'Strict-Transport-Security': 'max-age=31536000; includeSubDomains',
    'X-Content-Type-Options': 'nosniff',
    'X-Frame-Options': 'DENY'
  },
  enableCookieJar: true,
  // Additional security configurations automatically applied
});

License

MIT © Uniqhtt Contributors

Contributing

We welcome contributions! Please see our Contributing Guide for details on our development process, coding standards, and submission guidelines.

Uniqhtt - Where sophisticated HTTP client architecture meets elegant developer experience. From simple API integrations to complex web automation workflows, Uniqhtt scales with your requirements while maintaining enterprise-grade reliability and performance.

Readme

Keywords

Package Sidebar

Install

npm i uniqhtt

Weekly Downloads

19

Version

1.2.6

License

MIT

Unpacked Size

1.25 MB

Total Files

11

Last publish

Collaborators

  • yuniqsolutions
Try on RunKit
Report malware