Logs

Looking at logs takes time but can lead to valuable information.

Windows

  • Windows Event Logs - Wikipedia

    Windows logs a lot of information. It can be read using mmc.exe, under “Windows Logs”.

    The categories are:

    CategoryDescription
    ApplicationPrograms (started, stopped …)
    SecuritySecurity events (login, logout, …)
    SystemChanges to system (boot, shutdown, peripherals …)
    SetupSystem maintenance (update logs, …)

Linux

  • Linux logs - Wikipedia

    Linux logs are stored in /var/log/. The most important ones are:

    FileDescription
    auth.log or secureAuthentication events (login, logout, …)
    syslog or messagesGeneral messages (system wide)
    dpkg.logPackage management
    kern.logKernel messages
    btmpFailed login attempts
    wtmpLogin/logout history
    lastlogLast login for each user

    btmp, wtmp and lastlog can be read using last <file>

    Other applications can have their own logs in /var/logs.

Apache

  • Apache logs - Website

    Apache logs are often stored in /var/log/apache2/. The most important ones are:

    FileDescription
    access.logHTTP requests
    error.logHTTP errors
    other_vhosts_access.logHTTP requests from other virtual hosts

    access.log can be read using tail -f <file> or with grep to filter the logs.

    It can also be imported into a pandas dataframe using this snippet:

    # Read access.log file
    df = pd.read_csv(filename,
                sep=r'\s(?=(?:[^"]*"[^"]*")*[^"]*$)(?![^\[]*\])',
                engine='python',
                usecols=[0, 3, 4, 5, 6, 7, 8],
                names=['ip', 'datetime', 'request', 'status', 'size', 'referer', 'user_agent'],
                na_values='-',
                header=None
                    )
    
    # Extract the date from the datetime column
    df['date'] = df['datetime'].str.extract(r'\[(.*?):', expand=True)
    
    # Extract the time from the datetime column
    df['time'] = df['datetime'].str.extract(r':(.*?)\s', expand=True)