@skynetxbt/flow-hyperbrowser-research

Hyperbrowser Research Flow

A SkynetXBT flow for automated web research and data extraction using the Hyperbrowser plugin. This flow enables intelligent web research, content analysis, and automated data gathering with advanced LLM capabilities.

Features

• Automated web research with multiple LLM support
• Intelligent content extraction and analysis
• Vision-enabled research capabilities
• Multi-step research planning
• Real-time research monitoring
• Anti-detection stealth mode
• Automated session management

Installation

npm install @skynetxbt/flow-hyperbrowser-research
# or
yarn add @skynetxbt/flow-hyperbrowser-research

Dependencies

• @skynetxbt/core
• @skynetxbt/plugin-hyperbrowser
• @skynetxbt/agent

Usage

import { HyperbrowserResearchFlow } from '@skynetxbt/flow-hyperbrowser-research';

// Initialize with configuration
const flow = new HyperbrowserResearchFlow({
  llm: "claude-3-7-sonnet-20250219",
  validateOutput: true,
  useVision: true,
  maxSteps: 15,
  useVisionForPlanner: true,
  maxActionsPerStep: 10,
  maxInputTokens: 1000,
  plannerLlm: "gemini-2.0-flash",
  pageExtractionLlm: "gemini-2.0-flash",
  plannerInterval: 10,
  sessionOptions: {
    useStealth: true,
    adblock: true,
    trackers: true
  }
});

// Execute research
const result = await flow.execute({
  message: "research the latest developments in Ethereum L2 solutions"
});

// Monitor research progress
flow.on('researchComplete', (data) => {
  console.log('Research completed:', data);
});

Configuration Options

interface HyperbrowserConfig {
  // LLM Selection
  llm?: "claude-3-7-sonnet-20250219" | "gpt-4o" | "gemini-2.0-flash";
  plannerLlm?: "gemini-2.0-flash" | "claude-3-7-sonnet-20250219";
  pageExtractionLlm?: "gemini-2.0-flash" | "claude-3-7-sonnet-20250219";
  
  // Research Parameters
  maxSteps?: number;
  maxActionsPerStep?: number;
  maxInputTokens?: number;
  plannerInterval?: number;
  
  // Feature Flags
  validateOutput?: boolean;
  useVision?: boolean;
  useVisionForPlanner?: boolean;
  keepBrowserOpen?: boolean;
  
  // Session Configuration
  sessionOptions?: {
    useStealth?: boolean;
    adblock?: boolean;
    trackers?: boolean;
    solveCaptchas?: boolean;
    device?: ("desktop" | "mobile" | "tablet")[];
    platform?: ("chrome" | "firefox" | "safari" | "edge")[];
  };
}

Research Capabilities

Natural Language Research

The flow processes natural language research requests by extracting key topics and planning research steps:

// Example research queries
"fetch me good altcoins to invest in"
"research the top performing defi protocols"
"find information about new blockchain projects"
"analyze the market trends for NFTs"
"get info on upcoming token launches"

Research Monitoring

Monitor research progress with built-in status tracking:

const jobId = "your-research-job-id";

// Check research status
const status = await flow.getBrowserUseTask(jobId);
console.log('Research status:', status);

// Monitor progress
flow.on('pageAnalyzed', (data) => {
  console.log('Page analyzed:', data);
});

Error Handling

try {
  const research = await flow.execute({
    message: "analyze DeFi trends"
  });
} catch (error) {
  console.error('Research Error:', error);
}

// Configure retry behavior
flow.setRetryPolicy({
  maxRetries: 3,
  retryDelay: 5000
});

Best Practices

1. Session Management

   • Use stealth mode for sensitive research
   • Enable adblock and tracker blocking
   • Configure appropriate timeouts

2. Research Planning

   • Use clear, specific research queries
   • Set appropriate step limits
   • Configure vision capabilities as needed

3. Error Handling

   • Implement proper error monitoring
   • Set up retry policies
   • Handle timeouts gracefully

4. Resource Management

   • Clean up browser sessions
   • Monitor memory usage
   • Handle concurrent research tasks

Advanced Features

Vision-Enabled Research

The flow supports visual analysis of web content:

const flow = new HyperbrowserResearchFlow({
  useVision: true,
  useVisionForPlanner: true
});

Multi-LLM Pipeline

Configure different LLMs for various research stages:

const flow = new HyperbrowserResearchFlow({
  llm: "claude-3-7-sonnet-20250219",      // Main research
  plannerLlm: "gemini-2.0-flash",         // Step planning
  pageExtractionLlm: "gemini-2.0-flash"   // Content extraction
});

Session Customization

Fine-tune browser behavior:

const flow = new HyperbrowserResearchFlow({
  sessionOptions: {
    useStealth: true,
    device: ["desktop"],
    platform: ["chrome"],
    adblock: true,
    trackers: true
  }
});

Contributing

Contributions are welcome! Please read our contributing guidelines and submit pull requests to our repository.

License

MIT