BrowserReflect

BrowserReflect is a service designed to capture and display screenshots from various websites using a browser. It automates the process of monitoring website visuals, making it ideal for real-time data observation and presentation.

Overview

BrowserReflect consists of two main services:

UI: The UI service provides an HTTP interface that lists available screenshots and websites, enabling users to select and view the most recent captures with limited access. This setup is particularly useful for showcasing website updates in environments such as offices, control rooms, or public displays.
Worker: The worker service visits specified websites at regular intervals (configurable to suit your needs) and captures screenshots. This allows users to keep track of changes in website visuals without manual refreshing.

Features

BrowserReflect automates website screenshots with real-time updates, making it ideal for monitoring and visualizing site changes.

Key features include:

Automated Site Screenshots

Capture screenshots of specified websites at regular intervals, with customizable settings for each site. This feature is perfect for monitoring dashboards, web applications, and static pages.
Configurable Capture Intervals

Set custom reload and capture intervals for each site. You can configure these settings globally or individually, allowing flexibility based on your monitoring needs.
Real-Time UI for Viewing Screenshots

You can use a web-based interface to view the latest screenshots for each site. The UI updates dynamically, providing an up-to-date view of all monitored sites.
Multi-Site and Multi-Worker Support

BrowserReflect lets you monitor multiple sites using independent workers. Each worker can have unique configurations, such as capture intervals, login credentials, and browser settings.
Customizable Browser Options

Adjust browser parameters for each worker, including window size and scrollbar visibility. This flexibility ensures that screenshots are captured in the desired format for optimal visibility and detail.
Authentication for Protected Sites

Add credentials in the configuration to capture screenshots of sites that require authentication. BrowserReflect allows you to set credentials globally or for specific sites, making it suitable for secure or private content.
Flexible Configuration

Manage all settings, including site URLs, intervals, authentication, and UI options, through a single configuration file (reflect.yaml). This setup simplifies deployment and modifications without altering the application code.
Docker-Based Deployment

Run BrowserReflect efficiently with Docker, ensuring compatibility and simplified updates. Use Docker Compose to launch all required services in a single command, including Selenium for browser automation.
Detailed Logging

View real-time logs to monitor each worker’s actions and capture intervals and any errors. This logging helps you troubleshoot and confirm that the service operates as expected.
Scalable and Extensible

Scale BrowserReflect to monitor additional sites, handle higher capture frequencies, and manage more configurations. You can adapt the service for various use cases, from website monitoring to visual regression testing.

These features make BrowserReflect a powerful and flexible tool for automated website monitoring, giving you real-time insights and visualization across multiple web interfaces.

Installation

Clone the Repository

git clone https://github.com/railadeividas/BrowserReflect.git
cd BrowserReflect

Configure Intervals and Websites

Customize the configuration file to specify the list of websites and the intervals for screenshot updates.

See configuration/reflect.yaml-dist for examples.

Make your own copy cofiguration file to configuration/reflect.yaml.
Run BrowserReflect with Docker Containers

Make a copy of docker-compose.yml-dist to docker-compose.yml.

Edit configartion path in browserreflect-ui and browserreflect-worker.

Start BrowserReflect stack
```
docker-compose up -d
```

Configuration

Time Format

In BrowserReflect, time intervals can be specified using a shorthand format with units for days (d), hours (h), minutes (m), and seconds (s). The format combines these units without spaces, and you can include any combination of them in any order. Each unit represents a portion of time, allowing for precise interval settings.

Examples of valid formats:

0s – 0 second
1h3s – 1 hour and 3 seconds
30m2s – 30 minutes and 2 seconds
40s – 40 seconds
1d2m – 1 day and 2 minutes
1d4h2m2s – 1 day, 4 hours, 2 minutes, and 2 seconds

Each unit is optional, and if it’s not included, that portion is considered zero.

Configaration File

Configuration for BrowserReflect is managed through a configuration/reflect.yaml file. Key fields include:

logging_level: Specifies the logging level for the application. (Default: WARNING)

The available options are DEBUG, INFO, WARNING, ERROR, and CRITICAL.
global_worker_configuration: Defines the global settings for all workers, which can be overridden on a per-worker basis.
- selenium_hub_url: URL of the Selenium Hub used by workers to connect and automate the browser (e.g., http://selenium-hub:4444/wd/hub).
- data_path: Path to store captured screenshots and other data.
- browser_implicitly_wait: Time in seconds for implicit waits in the browser, allowing elements to load fully before actions are taken.
- browser_params: List of browser-specific parameters for headless operation, display size, or hiding elements like scrollbars.
- site_reload_interval: Specifies how frequently to reload the same URL, useful when the site’s content does not change dynamically.
  
  When multiple URLs are provided, this interval also determines how often the worker cycles to the next URL in the list.
- site_capture_interval: Interval in seconds between screenshots.
- site_login_credentials: Optional authentication details for sites requiring a login (to turn off authentication set null).
  - username: Username for the site.
  - password: Password for the site.
- site_login_delay: Time to wait after logging in before capturing the first screenshot to allow page load completion.
workers_configuration: List of worker configurations, each dedicated to capturing a specific site. Each worker can override global settings.
- site_name: Descriptive name for the site, helping to identify each capture task.
- site_url: URL or a list of URLs for the target site(s). For multiple URLs, BrowserReflect will rotate through them based on the specified site_reload_interval.
- selenium_hub_url: (Optional) Worker-specific Selenium Hub URL.
- data_path: (Optional) Worker-specific data storage path.
- browser_implicitly_wait: (Optional) Worker-specific wait time for page elements.
- browser_params: (Optional) Worker-specific browser parameters.
- site_reload_interval: (Optional) Worker-specific interval in seconds to reload the current URL.
  
  This is useful for capturing sites with static content that doesn’t change dynamically.
  
  If multiple URLs are provided, this interval also determines how often the worker cycles to the next URL.
- site_capture_interval: (Optional) Worker-specific capture interval.
- site_login_credentials: (Optional) Worker-specific login credentials (to turn off authentication set null).
- site_login_delay: (Optional) Worker-specific delay after login.

Example 1:

---
# General configuration
logging_level: WARNING

# Global worker configuration
global_worker_configuration:
  selenium_hub_url: http://selenium-hub:4444/wd/hub
  data_path: /app/data
  browser_implicitly_wait: 0s
  browser_params:
    - --window-size=1920,1580
    - --hide-scrollbars
  site_reload_interval: 1h
  site_capture_interval: 5s
  site_login_credentials:
    username: admin
    password: admin
  site_login_delay: 6s

# List of workers to capture sites
workers_configration:
  - site_name: Emoji in stat names1
    site_url: https://play.grafana.org/d/d625e719-aaca-4c2a-9ba5-4318f46f862a/emoji-in-stat-names?kiosk=1&refresh=5s

  - site_name: Time series features detailed overview1
    site_url: https://play.grafana.org/d/hxne1Hm4z/1-time-series-features-detailed-overview?kiosk=1&refresh=5s

  - site_name: Grafana with auth
    site_url: http://grafana:3000/d/dashboard1/1-a-view-of-nothing?kiosk=1&refresh=5s
    site_login_credentials:
      username: admin
      password: admin

  - site_name: Grafana with auth and multiple urls
    site_login_credentials:
      username: admin
      password: admin
    site_reload_interval: 30s
    site_capture_interval: 10s
    site_url:
      - http://grafana:3000/d/dashboard1/1-a-view-of-nothing?refresh=5s
      - http://grafana:3000/d/dashboard2/2-a-view-of-nothing?refresh=10s
      - http://grafana:3000/d/dashboard3/3-a-view-of-nothing?refresh=15s
      - http://grafana:3000/d/dashboard4/4-a-view-of-nothing?refresh=20s

  - site_name: Grafana without auth
    site_url: http://grafana:3000/d/dashboard2/2-a-view-of-nothing?kiosk=1&refresh=5s
    site_login_credentials: null

  - site_name: Site with all possible options
    site_url: http://grafana:3000/d/561sdf3dss/login-test?kiosk=1&refresh=5s
    selenium_hub_url: http://selenium-hub:4444/wd/hub
    data_path: /app/private
    browser_implicitly_wait: 0s
    browser_params:
      - --window-size=1920,1580
      - --hide-scrollbars
    site_reload_interval: 1h
    site_capture_interval: 5s
    site_login_credentials:
      username: admin
      password: admin
    site_login_delay: 6s

Adjust the configuration settings based on your specific requirements. This example shows two sites set up for capture, each with unique URLs. BrowserReflect will manage each worker individually, allowing for flexible and customizable site capture.

Example 2:

Minimal configuration with only two sites.

In this example Worker and UI will work with defaults.

---
# List of workers to capture sites
workers_configuration:
  - site_name: Local Grafana with auth
    site_url: http://grafana:3000/d/dashboard1/1-a-view-of-nothing?kiosk=1&refresh=5s
    site_login_credentials:
      username: admin
      password: admin

  - site_name: Emoji in stat names
    site_url: https://play.grafana.org/d/d625e719-aaca-4c2a-9ba5-4318f46f862a/emoji-in-stat-names?kiosk=1&refresh=5s

Usage

Once BrowserReflect is configured, you can start the service to begin capturing and viewing website screenshots. Follow these steps:

Start the Services

Run BrowserReflect using Docker Compose to initialize the UI and worker services along with any dependencies, like Selenium. From the project root directory, use:
```
docker compose up -d
```
This will start all configured services in detached mode.
Access the UI

The UI service is configured using environment variables in the docker-compose.yml file.

Key Environment Variables
- UI_HOST: The host IP to bind the UI service (default: 0.0.0.0).
- UI_PORT: The port to expose the UI service (default: 80).
- UI_WORKERS: Number of workers to handle UI requests (default: 4).
- UI_RELOAD: Whether to enable hot reload (default: False).
- UI_ACCESS_LOG: Whether to enable access logs (default: False).
Accessing the UI

Once the containers are up and running, you can access the UI in a web browser by navigating to:

http://<host>:<port>

Replace <host> with the address of the Docker host and <port> with the port configured in the environment variables (e.g., http://localhost:80).

The UI will display the list of configured sites and their latest captured screenshots. Site configurations are defined in the reflect.yaml file, which is loaded as part of the container's configuration.
View Real-Time Screenshots

Select any site from the list to see its most recent screenshot. Screenshots are updated based on the site_capture_interval configured for each worker. You can refresh the page to view the latest capture or set it to auto-refresh based on your browser settings.
Monitoring and Logs

To monitor the service or troubleshoot issues, view logs with:
```
docker compose logs -f
```
This will show real-time logs from all services, including worker activity, capture intervals, and any errors.
Stop the Services

To stop BrowserReflect and all related services, use:
```
docker compose down
```
This will cleanly shut down all services and release resources.
Updating Configuration

If you need to update the configuration file, make your changes in configuration/reflect.yaml and restart the services for changes to take effect:
```
docker-compose down
docker-compose up -d
```

Contributing

Contributions are welcome! Feel free to open issues, submit pull requests, or suggest features.

License

MIT License

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
build		build
configuration		configuration
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
docker-compose.yml-dist		docker-compose.yml-dist
use-puppeteer-readme.md		use-puppeteer-readme.md

Navigation Menu

Repository files navigation

BrowserReflect

Overview

Features

Installation

Configuration

Time Format

Configaration File

Example 1:

Example 2:

Usage

Contributing

License

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

BrowserReflect

Overview

Features

Installation

Configuration

Time Format

Configaration File

Example 1:

Example 2:

Usage

Contributing

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages