Uname : Linux premium36.web-hosting.com 4.18.0-553.44.1.lve.el8.x86_64 #1 SMP Thu Mar 13 14:29:12 UTC 2025 x86_64
Soft : LiteSpeed
Ip : 198.54.115.237
Port : 443
~
/
opt
/
cloudlinux
/
venv
/
lib
/
python3.11
/
site-packages
/
cl_website_collector
/
__pycache__
[ HOME ]
Exec
Submit
website_collector.cpython-311.pyc
� �8iZm � � � d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlmZmZ d dl m Z d dlmZm Z mZ d dlmZ d dlZd dlmZ d dlmZmZ d dlmZ d d lmZ d d lmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z% G d� d� � Z&dS ) � N)�datetime�timezone)�Path)�Dict�Any�Optional)�deepcopy)�PKG_VERSION)�cpusers�userdomains)�ApacheProcessor)�DocrootProcessor) �VERSION� MAJOR_VER�STORAGE_DIR�OPT_OUT_FILE�DOCROOT_SCAN_TIMEOUT�DOCROOT_LIMIT_PER_RUN�SENDING_TIMEOUT�SENDING_COMPRESSION�MAX_RECORD_SIZE�FILE_SIZE_LIMITc �v � e Zd ZdZd1dej dee dee fd�Z de de fd �Zde fd �Zd2d�Z d2d�Zd edee fd�Zd2d�Zdedefd�Zdededdfd�Zd2d�Zedefd�� � Zdeddfd�Zdededededede eef fd�Zde eef defd�Zde eef de eef fd �Zd ed!ed"ed#e eef de eef d$ede eef fd%�Zd&ed'ed(ed)edef d*�Zd&ed'edededededed+edefd,�Z ded-eded&ed'edededededed+edefd.�Z! d3dededededed&ed+ed'eddfd0�Z"dS )4�WebsiteCollectoraB Website data collection. Collects and exports various Apache-related data: - Apache configuration files (.conf) from standard directories - Apache system information (version, loaded modules) - .htaccess files from website document roots - Maintains processing state to avoid duplicate scans N�logger�storage_dir� max_sitesc �� � |�t }|| _ || _ | j � dt | j � � � � | � |�� � t | j �� � | _ t | j �� � | _ |dz | _ |dz | _ di dd �| _ t | j � � | _ | � � � | � � � dS ) a Initialize website collector for on-demand data collection and sending. Args: storage_dir: Directory for storing website collector state max_sites: Maximum number of sites to process (with hash-based selection) Nz1[WEBSITE-COLLECTOR] Initialized with max_sites=%s)r )r zwebsite_collector_stats.jsonz dry-run.jsonlFr )�global_apache_configs_sent�docroots�collected_version)r r r �debug�str�_ensure_directoriesr �docroot_processorr �apache_processor� stats_file�dry_run_file�sending_stat_defaultsr � sending_stats�_load_sending_stats�reset_cache_within_version)�selfr r r s �]/opt/cloudlinux/venv/lib64/python3.11/site-packages/cl_website_collector/website_collector.py�__init__zWebsiteCollector.__init__4 s� � � ��%�K����"�������M�s�SW�Sa�Ob�Ob�c�c�c� � � �[� �9�9�9� "2���!E�!E�!E��� /�t�{� C� C� C��� &�(F�F���'�/�9���*/��!"�& � & ��"� &�d�&@�A�A��� � � �"�"�"� �'�'�)�)�)�)�)� � docroots_data�returnc � � | j �|S t | j � di � � � � }t d| j |z � � }|dk r#| j � d|| j � � i S t |� � |k r|S g }|� � � D ]U\ }}t j |� � � � � � � � }|� |||f� � �V|� d� �� � i }|d|� D ]\ } }}|||<