compass.log_analyzor package

Submodules

compass.log_analyzor.adapter_matcher module

Module to provider installing progress update functions for the adapter.

class compass.log_analyzor.adapter_matcher.AdapterItemMatcher(file_matchers)

Bases: object

progress matcher for the os installing or package installing.

updateProgress(hostname, progress)

Update progress.

Args:
hostname: str, the hostname of the installing host. progress: Progress instance to update.
Returns:
None
update_progress_range(min_progress, max_progress)

update min_progress and max_progress.

class compass.log_analyzor.adapter_matcher.AdapterMatcher(name, os_pattern, os_matcher, min_os_progress, max_os_progress, package_installer_name, package_matcher, min_package_progress, max_package_progress)

Bases: object

Adapter matcher to update adapter installing progress.

classmethod getClusterProgress(clusterid)

Get cluster progress from database.

Args:
clusterid: int, cluster id for the cluster.
Returns:
Progress instance from database.

Notes: the function should be called out of database session.

classmethod getHostProgress(hostid)

Get Host Progress from database.

Args:
hostid: int, the id of host in database.
Returns:
Progress instance got from database or None if there is no such record.

Notes: The function should be called out of database session.

match(os_name, package_installer_name)
Check if the adapter matcher is acceptable for the given os name and package installer.
Args:
os_name: str, the os name. package_installer_name: str, the package installer name.
Returns:
True if found the AdapterMatcher can process the log files generated for the os installation and package installation.
classmethod updateClusterProgress(clusterid, progress)

update cluster installing progress to database.

Args:
clusterid: int, cluster id. progress: Progress instance to update to database.
Returns:
None

the progress will be updated to database if the value is greater than the progress in the database or the value is the same but the message is different.

Notes: the function should be called out of the database session.

classmethod updateHostProgress(hostid, progress)

update host progress to database.

Args:
hostid: int, the id of the ClusterHost in database. progress: Progress instance to update.
Returns:
None

the progress will be updated to database if the value is greater than the progress in the database or the value is the same but the message is different.

Notes: the function should be called out of the database session.

updateProgress(clusterid, hostids)

update cluster progress and hosts progresses.

Args:
clusterid: int, the cluster id. hostids: list of int, the host ids.
Returns:
None

compass.log_analyzor.file_matcher module

Module to update intalling progress by processing log file.

class compass.log_analyzor.file_matcher.CompositeFileFilter(filters)

Bases: compass.log_analyzor.file_matcher.FileFilter

filter log file based on the list of filters

appendFilter(file_filter)

append filter.

filter(pathname)

filter log file.

class compass.log_analyzor.file_matcher.FileFilter

Bases: object

base class to filter log file.

filter(pathname)

filter log file.

Args:
pathname: str, the absolute path name to the log file.
Returns:
None
class compass.log_analyzor.file_matcher.FileMatcher(line_matchers, min_progress, max_progress, filename)

Bases: object

File matcher the get the lastest installing progress from the log file.

updateProgress(hostname, total_progress)

update progress from file.

Args:
hostname: str, the hostname of the installing host. total_progress: Progress instance to update.
Returns:
None

the function update installing progress by reading the log file. It contains a list of line matcher, when one log line matches with current line matcher, the installing progress is updated. and the current line matcher got updated. Notes: some line may be processed multi times. The case is the last line of log file is processed in one run, while in the other run, it will be reprocessed at the beginning because there is no line end indicator for the last line of the file.

updateTotalProgress(file_progress, total_progress)

Get the total progress from file progress.

Args:
file_progress: Progress instance. the installing progress from
the logging file. It is between 0 to 1. 1 means the whole log installing progess for the related part is done. it does not mean the whole installing progress done.
total_progrss: Progress instance. the total installing
progress to update. It should be between absolute_min_progress and absolute max progress.
Returns:
None
update_absolute_progress_range(min_progress, max_progress)

update the min progress and max progress the log file indicates.

class compass.log_analyzor.file_matcher.FileReader(pathname)

Bases: object

Class to read log file.

The class provide support to read log file from the position it has read last time. and update the position when it finish reading the log.

getHistory()

Get log file read history from database.

Args:
None
Returns:
line_matcher_name: str, line matcher name. progress: Progress instance record the installing history.

The function should be called out of database session. It reads the log_progressing_history table to get the position in the log file it has read in last run, the partial line of the log, the line matcher name in the last run, the progress, the message and the severity it has got in the last run.

readline()

log line generator.

For the last line of the file, it may regenerate in the next line because there is no

for the last line.

updateHistory(line_matcher_name, progress)

Update log_progressing_history table.

Args:
line_matcher_name: str, the line matcher name. progress: Progress instance to record the installing progress.
Returns:
None

The function should be called out of database session. It updates the log_processing_history table.

class compass.log_analyzor.file_matcher.FileReaderFactory(logdir, filefilter)

Bases: object

factory class to create FileReader instance.

getFileReader(hostname, filename)

Get FileReader instance.

Args:
hostname: str, the hostname of installing host. filename: the filename of the log file.
Returns:
FileReader instance if it is not filtered. Otherwise None.
class compass.log_analyzor.file_matcher.FilterFileExist

Bases: compass.log_analyzor.file_matcher.FileFilter

filter log file if not exists.

filter(pathname)

filter log file.

compass.log_analyzor.file_matcher.getFileFilter()

get file filter

compass.log_analyzor.line_matcher module

Module about line matcher to get the progress when found match with a line of the log.

class compass.log_analyzor.line_matcher.IncrementalProgress(min_progress, max_progress, incremental_ratio)

Bases: compass.log_analyzor.line_matcher.ProgressCalculator

Class to increment the progress.

update(message, severity, progress)

update progress from message and severity.

class compass.log_analyzor.line_matcher.LineMatcher(pattern, progress=None, message_template='', severity=None, unmatch_sameline_next_matcher_name='', unmatch_nextline_next_matcher_name='', match_sameline_next_matcher_name='', match_nextline_next_matcher_name='')

Bases: object

Progress matcher for each line.

updateProgress(line, progress)

update progress by the line.

Args:
line: str, one line in log file to indicate the
installing progress. The line may be partial if the latest line of the log file is not the whole line. But the whole line may be resent in the next run.

progress: the Progress instance to update.

Returns:
None
class compass.log_analyzor.line_matcher.Progress(progress, message, severity)

Bases: object

Progress object to store installing progress and message.

class compass.log_analyzor.line_matcher.ProgressCalculator

Bases: object

base class to generate progress.

update(message, severity, progress)

interface to update progress by message and severity.

Args:
message: str, installing message. severity: str, installing severity.
Returns:
None
classmethod updateProgress(progress_data, message, severity, progress)
Update progress with the given progress_data, message and severity.
Args:

progress_data: float between 0 to 1. message: str, installing progress message. severity: installing message severity.

Should be in one of [‘ERROR’, ‘WARNING’, ‘INFO’]. ‘ERROR’ means there is some errors in installing.

progress: Progress instance to update.

Returns:
None
class compass.log_analyzor.line_matcher.RelativeProgress(progress)

Bases: compass.log_analyzor.line_matcher.ProgressCalculator

class to update progress to the given relative progress.

update(message, severity, progress)

update progress from message and severity.

class compass.log_analyzor.line_matcher.SameProgress

Bases: compass.log_analyzor.line_matcher.ProgressCalculator

class to update message and severity for progress.

update(message, severity, progress)

update progress from the message and severity.

compass.log_analyzor.progress_calculator module

module to provide updating installing process function.

compass.log_analyzor.progress_calculator.getAdapterMatcher(os_name, package_installer)

get adapter matcher by os name and package installer name.

compass.log_analyzor.progress_calculator.updateProgress(os_name, package_installer, clusterid, hostids)

Update adapter installing progress.

Args:
os_name: str, os name. package_installer: str, package installer name. clusterid: int, cluster id. hostids: list of int, hosts ids.
Returns:
None

Module contents