Skip to Content

Trex ccms_config.py file syntax error

Dear Experts,

im trying to enable the CCMS Monitoring for our Trex system.

im following the guide : Configuring Monitoring of Availability (Heartbeat/GRMG) :

im stuck at the step of :

xecute the following command to configure monitoring of availability (heartbeat/GRMG):

python ccms_config.py --grmg


as it gives me the error :

File "<stdin>", line 1

python ccms_config.py --grmg

i have not edited this file at all yet i cant know the root cause of the error and i have no python coding experince to debug the issue.

below is the script of the file :

#!/usr/bin/env python
#
#----------------------------------------------------------------#
# TREX CCMS configuration script
#
# Part of TREX SAPINST installation package
#
#
#
# COPYRIGHT SAP AG 2004
#
#----------------------------------------------------------------
# This script runs on Windows(TM) and on the supported TREX UNIX platforms;
# it must be run under sidadm permissions
#----------------------------------------------------------------
#
# This script installs the configuration files needed to enhance #the 
# supportability of TREX, as correlated to the term 'CCMS'.
#
# This means Paramater reporting, Process monitoring, Alert monitoring,
# GRMG Heart Beat.
#
# It is also used for deinstalling these features for one TREX instance.
# Herefore, the script 'ccms_deconfig.py' calls it with the '-u' #option.
#
# Related are the SAP agent programs 'saposcol' and 'sapccmsr'.
# It does no installation/deinstallation ot the SAP agents.
#
# The features listed above may be switched on by options, #without any option,
# all features are (de-)installed.
#
# All output is appended to a log file, the file name is also an #option.
#----------------------------------------------------------------
# The script is intended to be called by SAPINST on installation/upgrade time.
# The correlated 'ccms_deconfig's cript is intended to be called #by SAPINST
# at Deinstallation of a TREX instance.
#
# It may also be called by hand from the command line after the #installation.
#----------------------------------------------------------------#--------------
#
#
#
#----------------------------------------------------------------#--------------
# Used Environment Variables:
# 
# SAP_RETRIEVAL_PATH : the only hook which TREX instance the #script belongs to
#                      must be set
# DONTDOIT : if this variable is set, the script execution is #canceled (Skip())
# DEBUG: any value switches on trace output
# SAPINST_MESSAGE_DEVLOG_THRESHOLD: value 'trace' switches on #trace output
# the last variable is used by SAPINST
#----------------------------------------------------------------
#
#{([    these are for parentheses balancing check
#       the #{ and #} pairs are here for more security against #indent errors






# python imports
import types
import os, sys, time
import socket, traceback, re, getopt, popen2
import ConfigParser # ini file handler
# trex imports
import ConfigMgrPy  # for sapgparam() call


#----------------------------------------------------------------#--------------
#     Options: all option switches of this script
#---------------------------------------------------------------- 
class Options:
#{
    def __init__(self):
    #{
        self.longoptions = ["help", "debug", "logmon", "procmon", "grmg",
                            "ccms", "ccmsinst", "uninstall", "file=", "restart",
                            "register", "unregister", "superuser=",
                            "force", "fakeit"]
        self.options     = "Fh?dlrpgicuf:URS:"
        # some globally used flag settings:
        self.logfile     = None
        self.debugFlag   = 0
        self.fakeFlag    = 0


        self.removeFlag  = 0


        self.logmonFlag  = 0
        self.procmonFlag = 0
        self.grmgFlag    = 0
        self.ccmsFlag    = 0
        self.ccmsInstFlag= 0
        self.restartFlag = 0
        self.registFlag  = 0
        self.unregistFlag= 0
        self.forceFlag   = 0
        self.logDirFlag  = 1   # will be 0 on 700_COR
        self.superUser   = "root"
    #}


    def installing(self):
        return self.removeFlag == 0
    
    def check(self):
    #{
        try:
            switches, arguments = getopt.getopt(sys.argv[1:], self.options, self.longoptions)
        except getopt.GetoptError:
        #{
            # print help information and exit:
            print "Aborted on option error.\n"
            self.usage()
            sys.exit(2)
        #}
        switched = 0
        for option, arg in switches:
        #{
            if option in ("-h", "-?", "--help"):
                self.usage()
                sys.exit(1)
            if option in ("-f", "--file"):
                self.logfile = arg
            if option in ("-S", "--superuser"):
                self.superUser = arg
            if option in ("-d", "--debug"):
                self.debugFlag = 1
            if option in (      "--fakeit",):  # hidden 
                self.fakeFlag = 1


            if option in ("-u", "--uninstall"):
                self.removeFlag = 1


            if option in ("-l", "--logmon"):
                self.logmonFlag = 1
                switched = 1
            if option in ("-p", "--procmon"):
                self.procmonFlag = 1
                switched = 1
            if option in ("-g", "--grmg"):
                self.grmgFlag = 1
                switched = 1
            if option in ("-c", "--ccms"):
                self.ccmsFlag = 1
                switched = 1
            if option in ("-i", "--ccmsinst"):
                self.ccmsFlag = 1
                self.ccmsInstFlag = 1
                switched = 1


            if option in ("-R", "--register"):
                self.registFlag = 1
            if option in ("-U", "--unregister"):
                self.unregistFlag = 1


            if option in ("-r", "--restart"):
                self.restartFlag = 1
            if option in ("-F", "--force"):
                self.forceFlag = 1
        #}


        # exclusive: if ccmsinst given, do nothin else
        if self.ccmsInstFlag:
            self.logmonFlag, self.procmonFlag, self.grmgFlag  = 0,0,0


        # default: if no option is given: all has to be done, switch it on
        if not switched:
            self.logmonFlag, self.procmonFlag, self.grmgFlag = 1,1,1


        if self.registFlag and self.unregistFlag:
            print("Option error: options -R/--register and -U/--unregister are exclusive");
            sys.exit(1)
        if self.registFlag and not self.ccmsFlag:
            print("Option error: option -R/--register must be set together with -c/--ccms");
            sys.exit(1)
        if self.unregistFlag and not self.ccmsFlag:
            print("Option error: option -U/--unregister must be set together with -c/--ccms");
            sys.exit(1)
        if self.unregistFlag and not self.removeFlag :
            print("Option error: option -U/--unregister must be set together with -u/--uninstall");
            sys.exit(1)
                
    #}  check()


    def usage(self):
    #{
        print "Usage: ", sys.argv[0], "[options...]", """
        Manages TREX supportability features.
        Options are:
        -u        | --uninstall: uninstall yet installed supportability features
        -f <fn>   | --file=<fn>  write log output to given file <fn>
        -d        | --debug:     get more log output
                                 if no feature options are given, all of the following are switched on:
        -l        | --logmon:    manage log file monitoring templates
        -p        | --procmon:   manage process monotoring options used by saposcol
        -g        | --grmg:      manage GRMG-'Heart Beat' service
        -i        | --ccmsinst:  manage parameter reporting facility as plug-in to instance specific agent 'sapccmsr'
                                 the -i option disables the options -l -p -g and -b (used by installer)
        -c        | --ccms:      manage parameter reporting facility as plug-in to agent 'sapccmsr'
        -R        | --register   register sapccmsr on central system
        -U        | --unregister unregister sapccmsr from central system
        -S <super> | --superuser <super>  define superuser name: use <super> instead of 'root'"""
#DEL     if (os.name != "nt"):
        print     "        -r      | --restart:   (re-)start the Saposcol and Sapccmsr Services, if installed"
        print     "        -F      | --force:     install supportability related files even if saposcol/sapccmsr not yet installed"
    #}


#}  end Options






#------------------------------------------------------------------------------
#   Log: provides logging feature: write to stdout and into log file  
#------------------------------------------------------------------------------
class Log:
#{
    def __init__(self, tag, debugging=0, fileName=None):
    #{
        self.tagstring = tag
        self.debugging = debugging
        self.sep = "--------------------|--------|--------------------------------------------------"
        
        self.logFileName = "ccms_config.log"
        if not fileName is None:
            self.logFileName = fileName 
        try:
            self.logFile = open(self.logFileName, "a")
        except:
            self.logFile = None
            self.error("ATTENTION: log file could NOT be written: " + self.logFileName)
        var = os.getenv("SAPINST_MESSAGE_DEVLOG_THRESHOLD")
        if var:
            if var == "trace": self.debugging = 1
        if os.getenv("DEBUG"): self.debugging = 1
    #}


    def tag(self, tag):
        self.tagstring = tag
    def logging(self, severity, message):
    #{
        dt=time.localtime()
        datum="%02d-%02d-%02d %02d:%02d:%02d" % (dt[0], dt[1], dt[2], dt[3], dt[4], dt[5])
        try:
            if (self.logFile): 
                self.logFile.write( "%s |  %s | %s: %s \n" % ( datum, severity, self.tagstring, message ) )
        except:
            pass
        print                       "%s |  %s | %s: %s "   % ( datum, severity, self.tagstring, message )
        
    #}


    def separator(self):
    #{
        print self.sep
        try:
            if (self.logFile): 
                self.logFile.write(self.sep+'\n')
        except:
            pass
    #}


    def stack(self):
    #{
        tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
        for l in tr: self.logging("ERROR", l)
    #}
    def log(self, text):   self.logging( "INFO ", text )
    def debug(self, text):
        if self.debugging: self.logging( "DEBUG", text )
    def error(self, text): self.logging( "ERROR", text )
#}








#------------------------------------------------------------------------------
#     Abort: is used for finishing and returning 'ERROR' to SAPINST 
#------------------------------------------------------------------------------
class Abort(Exception):
#{
#    def __init__(self,args = None):
#        logtrace()
#        self.args=args
#
    def logging(self):
    #{
        log.stack()
        txt=""
        for t in self.args: txt += t
        log.error(txt)
    #}
#}
#------------------------------------------------------------------------------
#     Skip: is used for canceling script but returning result 'OK' to SAPINST
#------------------------------------------------------------------------------
class Skip(Exception):
#{
    def logging(self):
    #{
        log.stack()
        txt=""
        for t in self.args: txt += t
        log.error(txt)
    #}
#}


#------------------------------------------------------------------------------
#     Fake: test all stuff in /tmp/CCMS or C:\temp\CCMS instead of real places
#------------------------------------------------------------------------------
class TestFAKER:  
#                must copy all to this directory before start fake testing
#                this is /etc/rc?.d, /etc/init.d and /usr/sap/tmp/sapccmsr etc
#{
    def __init__(self, faking=0):
    #{
        self.path=""
        self.faking = faking | switches.fakeFlag
        if (self.faking):
        #{
            if os.name == "nt":
                self.path = r'c:\temp\CCMS'
            else:
                self.path = "/tmp/CCMS"


            log.log("Called for tests only: use %s as root of all installations" % self.path)
            if not os.access(self.path, os.F_OK):
                os.makedirs(self.path)
        #}
    #}
#}




class Daemon:
#{
    def __init__(self):
    #{
        log.debug("Initializing Daemon")
        self.httpd = False
        self.grmg = False
        self.names = [ "TREXNameServer", "TREXIndexServer", "TREXQueueServer", "TREXPreprocessor", "TREXCruiser" ]
        self.ports={}
        for n in self.names: self.ports[n] = -1 # -1 indicates a server not configured in Daemon.ini
        self.ports["never"] = -1
        self.inifile = os.path.join(trex.trexHome(), "TREXDaemon.ini")
        try:
            c = ConfigParser.ConfigParser()
            c.OPTCRE = re.compile(r'(?P<option>[^:=\s][^:=]*)(\s*(?P<vi>[:=])\s*)?(?P<value>.*)$')
            c.read([self.inifile])
            self.grmg = c.has_section("grmg")
            ps = []
            for i in c.items("daemon"):
                if i[0] == 'programs':
                    p = i[1]
                    ps = p.split(',')
                    for j in range(0, len(ps)):  ps[j] =ps[j].strip()
                    break
            for progsect in ps:
                log.debug("handle section " + progsect)
                port = 0
                name = 'never'
                try:
                    prog = c.get(progsect, "executable")
                    log.debug("handle program " + prog)
                    if prog == 'httpd':
                        self.httpd = True
                        continue
                    name = self.checkServer(prog)
                except: continue
                try:
                    args = c.get(progsect, 'arguments')
                    log.debug("arguments: " + args)
                    if '-port' in args:
                        words = args.split() # strips off extra blanks
                        if len(words) > 1: port = int(words[1])
                        log.debug("Found %s port %d" % ( name, port ))
                except: pass
                if self.ports[name] == -1: self.ports[name] = port # take only the fist server of same kind
        except:
           log.error("Cannot analyze " + self.inifile)
     #}
    def checkServer(self, name):
        app = '.x'
        if os.name.upper() == 'NT': app = '.exe'
        for n in self.names:
           if name  == n + app: return n
        return 'never'
    def append(self, lines):
        try:
            f = open(self.inifile, "a")
            for line in lines: f.write(line + '\n'); log.debug(self.inifile + " append: " + line)
            f.close()
        except:
            tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
            for l in tr: log.error(l)
            raise Abort("Could not append to " + self.inifile)


    def indexServerPort(self):   return self.ports["TREXIndexServer"]
    def preproServerPort(self):  return self.ports["TREXPreprocessor"]
    def queueServerPort(self):   return self.ports["TREXQueueServer"]
    def nameServerPort(self):    return self.ports["TREXNameServer"]
    def cruiserServerPort(self): return self.ports["TREXCruiser"]
    def webServerActive(self):   return self.httpd
    def grmgSection(self):       return self.grmg
#}   Daemon




def watchProcess(command):
#{
    output = []
    result = 0
    log.debug("Execute command: " + command)
    if os.name == 'nt':
    #{   here, class Popen4 is not defined
        reader, writer = popen2.popen4(command)
        line = reader.readline()
        while len(line):
            output.append(line)
            log.debug("-->> " + line.strip("\r\n"))
            line = reader.readline()
        writer.close();
        result = reader.close()
        if result is None: result = 0
    #}
    else:
    #{
        log.debug("call system[%s]" % command)
        pipes = popen2.Popen4(command);
        result = pipes.wait()
        while 1:
        #{
            line = pipes.fromchild.readline()
            if len(line) == 0: break
            output.append(line)
            log.debug("-->> " + line.strip("\r\n"))
        #}
    #}
    log.debug("Result %d" % result)
    if result != 0: log.log("Execution of command [%s] returned error code: %d" % ( command, result ))
    return ( output, result == 0 )
#}




#------------------------------------------------------------------------------
#     TrexHOME: all information relevant to actual TREX instance
#------------------------------------------------------------------------------
class TrexHOME:
#{
    def __init__(self):
    #{
        
        ######################################################################
        # CAVEAT: the SAP_RETRIEVAL_PATH is here expected to have
        #         <hostname> directory appended !!!!!
        ######################################################################
        self.home = os.getenv('SAP_RETRIEVAL_PATH')
        if self.home is None:
            raise Abort("SAP_RETRIEVAL_PATH is undefined!", "aborted")
        else:
            self.home = os.path.normpath(self.home)
            log.log("SAP_RETRIEVAL_PATH is [" + self.home + "]")
        self.host = socket.gethostname()
        ( self.main, host ) = os.path.split(self.home)
        if not ( host.lower() == self.host.lower()):
            log.error( "SAP_RETRIEVAL_PATH contains host %s differing from real hostname %s (ignored)" % ( host, self.host ) )
        self.home = os.path.join(self.main, self.host) # SAP_RETRIEVAL_PATH, 'hostname-directory'
        (siddir, self.instdir) = os.path.split(self.main)
        (self.usrsap, self.SID)     = os.path.split(siddir)
        # TODO have to check SID for [A-Z][A-Z0-9][A-Z0-9]
        # TODO have to check for SAP_RETRIEVAL_PATH like ../usr/sap/SID/TREX00
        log.log("SID: " + self.SID)
        log.log("instance dir: " + self.instdir)


        self.instance = self.main[-2 : ] # cut off last 2 digits of SAP_RETRIEVAL_PATH which are the instance no
        self.platform = sys.platform
        log.log("actual platform is " + self.platform)
        self.handler = FileHandler()
        self.handler.addPattern(r'^listenport[ \t]*=+[ \t]*([0-9]+)', r'\1')  # used for all <server>Port() functions below
        os.chdir(self.main);


        if os.name == 'nt':
        #{
            driveLetter = self.home[0]   # should be ok after normpath()
            checker = r'%s:\usr\sap' % driveLetter + os.path.sep + self.SID
            if not os.path.dirname(self.main) == checker:
                log.log("SAP_RETRIEVAL_PATH expected below %s, but found at %s -- please check if this is correct!" % (checker, self.main))
        #}
        else:
        #{
            if not os.path.dirname(self.main) == os.path.join("/usr/sap" , self.SID):
                 log.log("SAP_RETRIEVAL_PATH expected below /usr/sap, but found at: " + self.main + " please check if this is correct!")
        #}


        self.dir_profile  = ConfigMgrPy.sapgparam("DIR_PROFILE")
        self.dir_global   = ConfigMgrPy.sapgparam("DIR_GLOBAL")
        self.profile      = os.path.join(self.dir_profile, "%s_%s_%s" % (self.SID, self.instdir, self.host))
        self.startProfile = os.path.join(self.dir_profile, "START_%s_%s" % (self.instdir, self.host))
        if not os.path.exists(self.startProfile): self.startProfile = self.profile


    #}  end __init__


    def getSysdir(self):
        #
        # try reg.exe utility to read registry entry for sapstartsrv service
        service = "SAP%s_%s" % (self.SID, self.instance) # like SAPDEV_03
        cmd = "reg query HKLM\\SYSTEM\\CurrentControlSet\\Services\\%s /v ImagePath" % service
        output, rc = watchProcess(cmd)
        dir=""
        if rc and len(output):
            for line in output:
                i = line.find("pf=")
                if i != -1:
                   j=line.find("profile")
                   dir = line[i+4 : j-1]
                   log.log("found SYS directory out of registry: " + dir)
                   break
        #
        # if no reg.exe, or other error: try sapcontrol querying running sapstartsrv
        if not dir:   # try sapcontrol
            cmd = "sapcontrol -nr %s -function ParameterValue DIR_INSTALL" % self.instance
            output, rc = watchProcess(cmd)
            dir=""
            if rc and len(output) >= 6 and output[3].strip() == "OK":
               dir = output[5].strip()
               log.log("found SYS directory using sapcontrol: " + dir)
        if not dir:
            raise Abort( "Could not get SAPGLOBALHOST variable for instance profile access; Please start TREX instance" )
        return dir


    def addCcmsToProfile(self, registered):
    #{
        standalone = (not registered) and "-standalone" or ""
        nodaemon   = (os.name == "posix") and "-nodaemon" or ""
        ccmsrName  = os.path.join("$(DIR_EXECUTABLE)","sapccmsr$(FT_EXE)")
        handler    = FileHandler()
        handler.setPattern(r'^([ \t]*Start_Program_.*sapccmsr.*)')
        if not handler.matchFile(self.startProfile):
        #{
            number=0
            output=open(self.startProfile+".new","w+")
            for line in file(self.startProfile,"r").readlines():
            #{
                l = line.strip()
                if l.startswith("Start_Program_"):
                    n = int(l[14:16])
                    if n > number: number = n
                if line.startswith("ccms/enable_agent"):
                    log.debug("disable sapstartsrv from being MTE delivery agent")
                    line = "ccms/enable_agent = 0\n"
                output.write(line)
            #}
            startline = "Start_Program_%02d = local %s pf=%s -j2ee %s %s -DCCMS\n" % ( number+1, ccmsrName, self.profile, standalone, nodaemon )
            output.write("#-----------------------------------------------------------------------\n")
            output.write("# Start sapccmsr Agent\n")
            output.write("#-----------------------------------------------------------------------\n")
            output.write(startline)
            output.close()
            handler.renameFile(self.startProfile, self.startProfile + ".bak")
            handler.renameFile(self.startProfile+".new", self.startProfile)
        #}
        else:
            line = handler.matched.strip()
            number = line[14:16]
            startline = "Start_Program_%s = local %s pf=%s -j2ee %s %s -DCCMS" % ( number, ccmsrName, self.profile, standalone, nodaemon )
            handler.setPattern(r'^([ \t]*Start_Program_.*sapccmsr.*)', handler.doubleBS(startline))
            handler.addPattern("^ccms/enable_agent.*$",  "ccms/enable_agent = 0")
            #if line == startline: return # already found as needed
            # standalone flag has changed or other changes
            handler.convertFile(self.startProfile, self.startProfile + ".new")
            handler.renameFile(self.startProfile, self.startProfile + ".bak")
            handler.renameFile(self.startProfile+".new", self.startProfile)
    #}


    def delCcmsFromProfile(self):
    #{
        handler    = FileHandler()
        handler.setPattern(r'^([ \t]*Start_Program_.*sapccmsr.*)', r'#deactivated# \1')
        if handler.matchFile(self.startProfile):
            handler.convertFile(self.startProfile, self.startProfile+".new")
            handler.renameFile (self.startProfile, self.startProfile+".bak")
            handler.renameFile (self.startProfile+".new", self.startProfile)
    #}


    def check(self):
    #{
        if (os.name == "posix"):
        #{
            if not switches.ccmsInstFlag:
                if not (os.getuid() == 0):
                    if not testFAKER.faking:
                        pass  # raise Abort("You have to call this script with ROOT privileges") 
                    else:
                        log.log("TEST FAKING this script: accept to be non-root user")
        #}
        if not os.getenv("DONTDOIT") is None:
            raise Skip("%s cancelled because of DONTDOIT environment variabe being defined" % sys.argv[0])
    #}  end check


    def baseName(self):            return os.path.basename(self.home)
    def trexMain(self):            return self.main
    def trexHome(self):            return self.home  # SAP_RETRIEVAL_PATH, 'hostname-directory'
    def makePath(self, directory): return os.path.join(self.home, directory)
    def trexInstance(self):        return self.instance
    def trexHost(self):            return self.host
    def trexExe(self):             return os.path.join(self.main, "exe")
    def trexSid(self):             return self.SID
    def usrSap(self):              return self.usrsap
    def instanceProfile(self):     return self.profile
    def trexTemplates(self):       return os.path.join(self.dir_global,"trex","templates")


    def indexServerPort(self):
        return self.handler.extractString(os.path.join(self.trexHome(), "TREXIndexServer.ini"))
    def queueServerPort(self):
        return self.handler.extractString(os.path.join(self.trexHome(), "TREXQueueServer.ini"))
    def preproServerPort(self):
        return self.handler.extractString(os.path.join(self.trexHome(), "TREXPreprocessor.ini"))
    def nameServerPort(self):
        return self.handler.extractString(os.path.join(self.trexHome(), "TREXNameServer.ini"))
    def cruiserServerPort(self):
        return self.handler.extractString(os.path.join(self.trexHome(), "TREXCruiser.ini"))
        
#}  end TrexHOME




#------------------------------------------------------------------------------
#   Agents: information related to CCMS agents saposcol and sapccmse: paths
#------------------------------------------------------------------------------
class Agents:
#{
    def __init__(self):
    #{
        log.tag("Agents")
        log.debug("initializing")
        self.host = socket.gethostname()
        self.agentBin    = trex.trexExe()  # "/usr/sap/<SID>/TRX<instance>/exe"
        self.agentHome   = testFAKER.path + trex.usrSap()   # d:\usr\sap


        if os.name == 'nt':
            nul = " 2>nul"
            exe=".exe"
        else:
            nul = " 2>/dev/null"
            exe=""


        self.ccmsDir     = os.path.join( self.agentHome, "ccms")
        cmd = "sappfpar pf="+trex.instanceProfile()+" DIR_CCMS"+nul
        output, rc = watchProcess(cmd)
        if rc and len(output):
                self.ccmsDir = output[0]
                log.debug("got ccmsDir: " + self.ccmsDir)
        self.ccmsDir     = os.path.join( self.ccmsDir, trex.trexSid() + "_" + trex.trexInstance())
        self.sapccmsr    = os.path.join( self.ccmsDir, "sapccmsr" ); # directory /usr/sap/ccms/SID_01/sapccmsr


        self.saposcolExe = os.path.join(self.agentBin, "saposcol"+exe)
        self.sapccmsrExe = os.path.join(self.agentBin, "sapccmsr"+exe)
        self.procmon     = os.path.join(self.ccmsDir, "procmon")
        self.logmon      = os.path.join(self.ccmsDir, "logmon")
        self.grmg        = os.path.join(self.ccmsDir, "grmg")


        self.lastTrexMonitored = 0    # flag if thes actual TREX is the last found in paramter reporting
        self.saposcolFound = 0
        self.sapccmsrFound = 0
        self.stoppedSapccmsr = 0      # we did not stop the service ( 1 means: yes we did it)
        self.saposcolStarted = 0
        self.sapccmsrStarted = 0


        self.handler = FileHandler()
        self.handler.setPattern( r'^([ \t]*DIR_LOGGING.*)')
        if self.handler.matchFile(trex.instanceProfile()):
            self.loggingDirDefined=1
            # retrieve the DIR_LOGGING value: the user may have entered another directory as we do here
            cmd = "sappfpar pf="+trex.instanceProfile()+" DIR_LOGGING"+nul
            output, rc = watchProcess(cmd)
            if rc and len(output):
                self.dirLogging = output[0]
                log.log("CCMS files will be hosted in: " + self.dirLogging)
                self.settings()
        else:
            self.loggingDirDefined=0
            self.dirLogging = os.path.join(trex.trexHome(), "log")
            log.debug("dirLogging: " + self.dirLogging)
    #}  end __init__


    # correct some file locations
    def settings(self):
    #{
        log.debug("settings called")
        if os.name == 'nt':
            self.sapccmsr    = os.path.join(self.dirLogging, "sapccmsr")  # like /usr/sap/<SID>/TRX01/<host>/log/sapccmsr
            self.logmon      = os.path.join(self.dirLogging, "logmon")
            self.grmg        = os.path.join(trex.trexHome(),"trace","grmg") # here, sapccmsr searches really for GRMG upload files
        else: # UNIX:
#            if not (switches.ccmsInstFlag):
#                self.agentBin    = self.agentHome + "/ccms/bin"   # "/usr/sap/ccms/bin"
#                self.sapccmsr    = self.agentHome + "/tmp/sapccmsr"
            self.sapccmsr    = os.path.join(self.dirLogging, "sapccmsr")  # like /usr/sap/<SID>/TRX01/<host>/log/sapccmsr
            self.logmon      = os.path.join(self.dirLogging, "logmon")
            self.grmg        = os.path.join(trex.trexHome(),"trace","grmg") # here, sapccmsr searches really for GRMG upload files
    #}  end settings()


    def check(self):
    #{
        if os.access(self.saposcolExe, os.F_OK|os.X_OK): self.saposcolFound = 1
        if os.access(self.sapccmsrExe, os.F_OK|os.X_OK): self.sapccmsrFound = 1
        if not switches.installing(): return
        if not os.access(self.agentHome, os.F_OK):
            log.log("CCMS agents home directory %s not yet existing, will be generated" % self.agentHome)
        if not os.access(self.sapccmsr, os.F_OK):
            log.log("CCMS directory %s not yet existing, will be generated" % self.sapccmsr)
        if switches.ccmsInstFlag: return 1
        if not os.access(self.logmon, os.F_OK):
            log.log("CCMS 'logmon' directory %s not yet existing, will be generated" % self.logmon)
        if not os.access(self.grmg, os.F_OK):
            log.log("CCMS 'grmg' directory %s not yet existing, will be generated" % self.grmg)
#DEL    if not os.access(self.saposcolExe, os.F_OK|os.X_OK):
#DEL        log.log("Hint: CCMS agent '%s' not yet installed, you may do it later on" % self.saposcolExe)
#DEL    if not os.access(self.sapccmsrExe, os.F_OK|os.X_OK):
#DEL        log.log("Hint: CCMS agent '%s' not yet installed, you may do it later on" % self.sapccmsrExe)
        return 1
    #}


    def checkDir(self,dir):
        if os.name == "posix":
            if not os.path.exists(dir):
                parent=os.path.dirname(dir)
                if not os.path.exists(parent): self.checkDir(parent) # now exists
                if not self.writeable(parent): self.sucommand("mkdir " + dir + "&& chmod a+w " + dir)
                else: os.makedirs(dir)
            else:
                if not self.writeable(dir): self.sucommand("chmod a+w "+dir)
                return
        else:
            if not os.path.exists(dir):
                os.makedirs(dir)


    def writeable(self,dir):
        if os.access(dir,os.W_OK): return 1
        s = os.stat(dir)
        if s.st_uid == os.getuid():
           try:
               os.chmod(dir, s.st_mode | stat.S_IWUSR)
               return 1
           except:
               log.debug("chmod exception: "+str(sys.exc_value))
               return 0
        else:
           return 0


    def sucommand(self,command):
        log.log("This command has to be called with superuser privileges: " +command)
        log.log("Enter the password of superuser '%s'" % switches.superUser)
        cmd = "su %s -c '%s'" % (switches.superUser, command)
        r = os.system(cmd)
        if r != 0: raise Abort("su command [%s] failed" % cmd)
   


    def addDirLogging(self):
    #{
        if not switches.logDirFlag or self.loggingDirDefined: return
        try:
            pf = open(trex.instanceProfile()+".new", "w+")
            d=0
            for line in file(trex.instanceProfile(),"r").readlines():
            #{
                pf.write(line)
                if line.startswith("DIR_") and not d:
                    d=1
                elif d==1:
                    # this equals to /usr/sap/SID/TREX00/<host>/log == trex.trexHome() + "/log"
                    line = "DIR_LOGGING=$(DIR_INSTANCE)"+os.sep+"$(SAPLOCALHOST)"+os.sep+"log"+"\n"
                    pf.write(line)
                    log.log("write DIR_LOGGING to " + trex.instanceProfile())
                    log.debug("line: " + line)
                    d=2
            #}
            pf.close()
            self.handler.renameFile(trex.instanceProfile(), trex.instanceProfile()+".bak")
            self.handler.renameFile(trex.instanceProfile()+".new", trex.instanceProfile())


            old = self.sapccmsrPath()
            self.settings()
            
            for f in ( "csmconf", "passwd"):
                src=os.path.join(old,f)
                dst=os.path.join(self.sapccmsrPath(),f)
                if os.path.exists(src):
                    if not os.path.exists(dst) or self.handler.isNewer(src,dst): self.handler.copyFile(src,dst)
        except:
            tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
            for l in tr: log.error(l)
            raise Abort("Cannot add DIR_LOGGING to instance profile %s" % trex.instanceProfile())
    #}


    def run(self):
    #{
        pass
#        if not os.access(self.agentBin, os.F_OK):
#            os.makedirs( self.agentBin)
#        if not os.access(self.sapccmsr, os.F_OK):
#            os.makedirs( self.sapccmsr)
#        if not os.access(self.logmon, os.F_OK):
#            os.mkdir(    self.logmon)
#        if not os.access(self.grmg, os.F_OK):
#            os.mkdir(    self.grmg)
    #}




    def saposcolRunning(self):
    #{
        output, rc = watchProcess(self.saposcolExe + " -s")
        if len(output) > 2 and output[2][0:9] == "  running":
            log.debug("saposcol seams to be running")
            return 1
        return 0
    #}


    def stopSapccmsr(self):
    #{
        # stop and start sapccmsr via command line
        if self.sapccmsrFound:
        #{
            self.stoppedSapccmsr = 1 # may it restart later
            log.log("Try to stop %s - please wait" % self.sapccmsrExe)
            watchProcess(self.sapccmsrExe + " pf=" + trex.instanceProfile() + " -j2ee -stop")
            time.sleep(7)
        #}end if 
    #}  end stopSapccmsr


    # start the sapccmsr, but only if we have it stopped. or if required
    def restartSapccmsr(self):
    #{
        log.debug("stopped: %d  restart: %d" % (self.stoppedSapccmsr, switches.restartFlag))
        if self.stoppedSapccmsr == 0 and switches.restartFlag == 0: return
        # yes we did stop it and may restart it


        prefix = (os.name=='nt') and 'start "sapccmsr" ' or ""
        log.log("Try to start %s" % self.sapccmsrExe)
        standalone = (not agents.registered()) and "-standalone" or ""
        cmd =prefix + self.sapccmsrExe + " pf=" + trex.instanceProfile() + standalone + " -j2ee -DCCMS"
        if not os.system(cmd):
            self.sapccmsrStarted = 1
    #} end restartSapccmsr


    # restart the 'saposcol' agent; check if running before
    def restartSaposcol(self):
    #{
        if not self.saposcolFound: return # sorry, but we have no access to the programs
        if self.saposcolStarted:         return # we have it alredy restarted, dont do it twice
        wasRunning = self.saposcolRunning()
        if wasRunning:
            watchProcess(self.saposcolExe + " -k")  # kill
        if switches.restartFlag or wasRunning:
            watchProcess(self.saposcolExe + " -l")  # launch
            self.saposcolStarted = 1
    #}


    #restart the agents, if they have been found
    def restart(self):
    #{
        log.tag("Agents")
        if self.saposcolFound and not self.saposcolStarted: self.restartSaposcol()
        if self.sapccmsrFound and not self.sapccmsrStarted:
        #{
            self.stopSapccmsr()    # stops it only if already running
            self.restartSapccmsr()
        #}
    #}


    def registerCcms(self):
    #{
        # if on other host registration hast been done, use their connection config files
        for f in ( "csmconf", "passwd"):
            src=os.path.join(trex.trexTemplates(),f)
            dst=os.path.join(agents.sapccmsrPath(),f)
            if os.path.exists(src):
                if not os.path.exists(dst) or self.handler.isNewer(src,dst): self.handler.copyFile(src,dst)
        log.log("Notice: answer to question '[optional] monitored system I28 belongs to system group' must be 'TREX_Systems'")
        cmd = self.sapccmsrExe + " pf=" + trex.instanceProfile() + " -j2ee -R"
        if os.name=='nt': cmd += " -noservice"  # avoid sapccmsr to register itself as windows service
        log.debug("Executing system(%s)" % cmd)
        if not os.system(cmd):
            log.log("sapccmsr has been registered")
            # copy connection files generated to global location for registration later on evenutally clones
            # saprc.ini is created on registration, need not ot be copied
            for f in ( "csmconf", "passwd"):
                src=os.path.join(agents.sapccmsrPath(),f)
                dst=os.path.join(trex.trexTemplates(),f)
                if os.path.exists(src): self.handler.copyFile(src,dst)
        # not more needed because of '-noservice' option:
        #if os.name=='nt':
        #    self.stopSapccmsr() 
        #    service="SAPCCMSR.%s" % trex.trexInstance()
        #    log.log("Sapccmsr for TREX cannot run as windows service, thus removing it from services list")
        #    watchProcess("sc delete " + service)
    #}
    def registered(self): # if csmconf and saprfc.ini exist, sapccmsr has been registred
        return os.path.exists(os.path.join(agents.sapccmsrPath(),"csmconf")) and os.path.exists(os.path.join(agents.sapccmsrPath(),"saprfc.ini"))


    def unregisterCcms(self):
        cmd = self.sapccmsrExe + " pf=" + trex.instanceProfile() + " -j2ee -U"
        log.debug("Executing system(%s)" % cmd)
        if not os.system(cmd):
            log.log("sapccmsr has been unregistered")




    def grmgPath(self):     return self.grmg
    def sapccmsrPath(self): return self.sapccmsr
    def logmonPath(self):   return self.logmon
    def procmonPath(self):  return self.procmon
    def sapccmsrIni(self):  return os.path.join(self.sapccmsr, "sapccmsr.ini")


    def notInstalled(self):       return self.saposcolFound == 0 or self.sapccmsrFound == 0
    def setIsLastMonitored(self): self.lastTrexMonitored = 1
    def getIsLastMonitored(self): return self.lastTrexMonitored == 1
#}  end Agents




#------------------------------------------------------------------------------
#   FileHandler: all file handling, text pattern matching, text file changes
#------------------------------------------------------------------------------
_readFileFlags = os.O_RDONLY
_readFileMode = 0444
_writeFileFlags = os.O_CREAT | os.O_WRONLY | os.O_TRUNC
_writeFileMode = 0666
_copyFileBuffSize = 102400 # 100k


if os.__dict__.has_key('O_BINARY'):
    _readFileFlags |= os.O_BINARY
    _writeFileFlags |= os.O_BINARY


class FileHandler:
#{
    def __init__(self):
    #{
        self.patterns     = []
        self.replacements = []
        self.regexps      = []
        self.ONLY_ONCE    = 1
        self.bsbs=        re.compile(r'[\\]') # see doubleBS
    #}


    #-- doubleBS --  redouble all backslashes
    # special handling of f''ing backslash handling on re.sub: doubl'em all before:
    def doubleBS(self, str):
        return self.bsbs.sub('\x5C\x5c\x5C\x5c', str) # the string means really '\\\\'


    def addPattern(self, pattern, replacement = None):
    #{
        log.debug("Pattern [%s]" % pattern)
        log.debug("Replace [%s]" % replacement)
        self.patterns.    append(pattern)
        self.replacements.append(replacement)
        self.regexps.     append(re.compile(pattern))
    #}
    def setPattern(self, pattern, replacement = None):
    #{
        log.debug("Pattern [%s]" % pattern)
        log.debug("Replace [%s]" % replacement)
        self.patterns =     [ pattern ]
        self.replacements = [ replacement ]
        self.regexps =      [ re.compile(pattern) ]
    #}


    #-- convertFile --
    # copy text files, replace lines if pattern and replacement(s) defined
    # switch off pattern matching, if onlyOnce != 0 and one pattern matched
    def convertFile(self, inFileName, outFileName = None, onlyOnce = 0):
    #{
        if outFileName is None:
        #{
            outFileName = inFileName
            inFileName = inFileName + ".bak"
            self.renameFile(outFileName, inFileName)
        #}
        log.log("Transpose " + inFileName + " to " + outFileName)
        inFile = None;
        outFile = None;
        found = 0
        try:
        #{
            inFile = open(inFileName, "r")
            outFile = open(outFileName, "w+")
            matching = (len(self.patterns) > 0)
            while 1:
            #{
                line = inFile.readline()
                if line == "": break   # eof reached
                if matching:
                #{
                    for index in range(0, len(self.patterns)): 
                    #{
                        tupel = self.regexps[index].subn(self.replacements[index], line, 1)
                        if tupel[1]:
                        #{
                            line = tupel[0]
                            log.debug("    replace line with [" + self.patterns[index] + "] by " + line[0:-1])
                            found = 1
                            if (onlyOnce): matching = 0
                            break
                        #}
                    #}
                #}
                outFile.write(line)
            #}
            inFile.close()
            outFile.close() 
        #}
        except Exception,e :
            if inFile != None: inFile.close()
            if outFile != None: outFile.close()
            tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
            for l in tr: log.error(l)
            raise Abort("FileHandler aborted")
        return found
    #}  end convertFile


    def matchFile(self, inFileName): # find only the first matching line
    #{
        log.debug("Match file " + inFileName )
        self.matched = ""
        result = 0
        inFile = None
        try:
        #{
            if not os.access(inFileName, os.F_OK): return 0
            inFile = open(inFileName, "r")
            matching = (len(self.patterns) > 0)
            while 1:
            #{
                line = inFile.readline()
                if line == "": break   # eof reached
                if matching:
                    for index in [0, len(self.patterns) - 1]: 
                    #{
                        if self.regexps[index].match(line):
                            log.debug("    found [" + self.patterns[index]+ "] in line [" + line[0:-1] + "]")
                            self.matched = line
                            result = 1
                            break
                    #}
                if (result == 1): break
            #}
            inFile.close()
        #}
        except:
            if inFile != None: inFile.close()
            tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
            for l in tr: log.error(l)
            raise Abort("FileHandler aborted")
        return result
    #}




    #-- purgeFile --
    # copy text files, omit lines where the pattern match
    # switch off pattern matching, if onlyOnce != 0 and one pattern matched
    def purgeFile(self, fileName, onlyOnce = 0):
    #{
        log.log("Purge file " + fileName )
        inFileName = fileName + ".bak"
        outFileName = fileName


        self.renameFile(fileName, inFileName);


        inFile = None;
        outFile = None;
        found = 0
        try:
        #{
            inFile = open(inFileName, "r")
            outFile = open(outFileName, "w+")
            matching = (len(self.patterns) > 0)
            while 1:
            #{
                line = inFile.readline()
                if line == "": break   # eof reached
                if matching:
                #{
                    found = 0
                    for index in [0, len(self.patterns) - 1]: 
                    #{
                        if self.regexps[index].match(line):
                        #{
                            log.debug("    found [" + self.patterns[index]+ "] in line [" + line[0:-1] + "]")
                            found = 1
                            if (onlyOnce): matching = 0
                            break
                        #}
                    #}
                #}
                if not found: outFile.write(line)
            #}
            inFile.close()
            outFile.close() 
        #}
        except Exception,e :
            if inFile != None: inFile.close()
            if outFile != None: outFile.close()
            tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
            for l in tr: log.error(l)
            raise Abort("FileHandler aborted")
        return found
    #}  end convertFile




    def extractString(self, inFileName): # find only the first matching line
    #{
        log.debug("Extract string from file %s" % inFileName)
        result = None
        inFile = None
        try:
        #{
            if not os.access(inFileName, os.F_OK): return None
            inFile = open(inFileName, "r")
            matching = (len(self.patterns) > 0)
            while result == None:
            #{
                line = inFile.readline()
                if line == "": break   # eof reached
                if matching:
                    for index in [0, len(self.patterns) - 1]: 
                    #{
                        tupel = self.regexps[index].subn(self.replacements[index], line, 1)
                        if tupel[1]:
                        #{
                            result = tupel[0]
                            if result[-1] == "\n": result = result[0:-1]
                            log.debug("    extract line with [" + self.patterns[index] + "] : [" + result + "]")
                            break
                        #}
                    #}
            #}
            inFile.close()
        #}
        except:
            if inFile != None: inFile.close()
            tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
            for l in tr: log.error(l)
            raise Abort("FileHandler aborted")
        return result
    #}


    # do a binary copy !
    def copyFile (self, inFileName, outFileName):
    #{
        log.debug("Copy file %s to %s" % ( inFileName, outFileName ) )
        # no exceptions caught!
        fdFrom = -1
        fdTo = -1
        try:
            fdFrom = os.open(inFileName, _readFileFlags, _readFileMode)
            if fdFrom == -1:
                raise Abort, 'Cannot open file "%s" for reading' % inFileName
            fdTo = os.open(outFileName, _writeFileFlags, _writeFileMode)
            if fdTo == -1:
                raise Abort, 'Cannot open file "%s" for writing' % outFileName
            while 1:
                s = os.read(fdFrom, _copyFileBuffSize)
                if os.write(fdTo, s) != len(s):
                    raise Abort, 'Error writing to file "%s"' % fdTo
                if len(s) != _copyFileBuffSize:
                    break
        finally:
            if fdFrom != -1:
                os.close(fdFrom)
            if fdTo != -1:
                os.close(fdTo)
                st = os.stat(inFileName)
                os.utime(outFileName, (st.st_atime, st.st_mtime))
        return
    #}


    def renameFile(self, inFileName, outFileName):
    #{
        try:
            if (os.access(outFileName, os.F_OK)):
            #{
                os.remove(outFileName)  # in order to avoid  OSError on already existent outFileName
            #}
            os.rename(inFileName, outFileName)
            log.debug("renamed %s to %s" % ( inFileName, outFileName)) 
        except:
            raise Abort("Could not rename " + inFileName + " to " + outFileName)
    #}


    def removeFile(self, fileName, mustExist=None):
    #{
        log.debug("Removing file %s" % fileName)
        if not os.access(fileName, os.F_OK):
            if (mustExist): Abort("File %s must exist" % fileName)
        else:
            os.remove(fileName)
            log.log("Removed file %s" % fileName)
    #}


    def appendToFile(self, fileName, text):
    #{
        log.debug("Append to file " + fileName)
        f = None
        try:
            f = open(fileName, "a")
            log.log("Append to %s line [%s]" % (fileName, text))
            f.write(text)
            f.write("\n")
            f.close()
        except:
            if not f is None: f.close
            raise
    #}


    def existFile(self, fileName):
    #{
        log.debug("Check existence of %s" % fileName)
        result = 0
        try:
            result = os.access(fileName, os.F_OK)
        except: 
            raise Abort("Error in file " + fileName)
        return result
    #}


    def makeLink(self, existing, target):
    #{
        already=0
        try:
            fs = os.lstat(target) # we are really on UNIX!
            already=1
        except:
            pass
        if already: # we are shure, the file exists, "and hopefully is no directory"
            log.debug("Symblic link %s already existing, removing first." % target)
            os.remove(target)
        os.symlink(existing, target)
    #}


    # compare modification date; src, dst have to exist
    def isNewer(self,src,dst):
    #{
        s = os.stat(src)
        d = os.stat(dst)
        return s.st_mtime > d.st_mtime
    #}


#}  end FileHandler












#------------------------------------------------------------------------------
#     ManageCCMS: install Parameter Monitoring feature 
#------------------------------------------------------------------------------
class ManageCCMS:
#{
    def __init__(self):
    #{
        log.tag("Manage CCMS")
        log.debug("initializing")
        self.handler = FileHandler()
        if os.name == "nt":
            self.sharedLib = "TrexCCMS.dll"
        else:
        #{
            if os.uname()[0] == "HP-UX":
                self.sharedLib = "TrexCCMS.sl"
            else:
                self.sharedLib = "TrexCCMS.so"
        #}
        self.notmore  = """# TREX Plugin is not more needed, its work is done by TREXNameServer, sapccmsr is for delivery to R/3"""
        self.template = """
### Configuration file for CCMS agents SAPCCMSR, SAPCM3X and SAPCCM4X
###
### Format of entries for plugins:
# PlugIn <file> trace
#
""" + self.notmore + """
# PlugIn %s
#
###
###
### Format of entries for logfile monitoring:
#  LogFile <full path of logfile template>
###
###
### Format of entries for the option to delete trees if no corresponding logfile exists:
### This Parameter is optional, if not specified the tree still remains
#  LogFileParam DelTree
###
###
### Format of entries for mechanism to filter out SAPOSCOL values:
# OsColFile <full path of oscolfile template>
#
### Do not monitor Java instance
Suppress aljsflib
.
###
""" % os.path.join(trex.trexMain(), "exe", self.sharedLib)


    #}  end __init__




    def check(self):
        log.debug("switched " + ("off", "on")[switches.ccmsFlag])
        return switches.ccmsFlag


    def run(self):
        log.log(" ")
        if switches.installing(): self.install()
        else:                     self.uninstall()


    def install(self):
    #{
        log.log("Installing CCMS parameter reporting")


        #-- add DIR_LOGGING to instance profile, if missing
        agents.addDirLogging()


        #-- handle sapccmsr.ini - the agent configuration file
        sharedlib  = os.path.join(trex.trexExe(), self.sharedLib)
        configHandler = FileHandler()
        if not os.access(agents.sapccmsrIni(), os.F_OK):
        #{
            # install our sapccms.ini template
            agents.checkDir(agents.sapccmsrPath())
            of = open(agents.sapccmsrIni(), "w+")
            of.write(self.template)
            of.close()
        else:
        #   check if the agent plug-in is already inside the sapccmsr.ini file
            configHandler.addPattern(r'^([ \t]*PlugIn)')
            if configHandler.matchFile(agents.sapccmsrIni()):
                configHandler.setPattern(r'^([ \t]*PlugIn.*)', self.notmore+"\n# PlugIn " + configHandler.doubleBS(sharedlib))
            else:
            #{
                configHandler.setPattern( r'^(#[ \t]*PlugIn <file>.*)',
                                          "# PlugIn <file> trace # "+self.notmore ) # PlugIn " + configHandler.doubleBS(sharedlib))
            #}
            configHandler.renameFile(agents.sapccmsrIni(), agents.sapccmsrIni() + ".bak")
            configHandler.convertFile(agents.sapccmsrIni() + ".bak", agents.sapccmsrIni(), self.handler.ONLY_ONCE); 
        #}
        if switches.registFlag:
            agents.registerCcms()
        trex.addCcmsToProfile(agents.registered())
        configHandler.setPattern(r'^ccms/enable_agent.*',r'ccms/enable_agent=1')
  sapprofile = os.path.join(trex.trexHome(),"sapprofile.ini")
  configHandler.convertFile(sapprofile)
        log.log("CCMS parameter reporting installed.")
    #} install()


    def uninstall(self):
    #{
        log.log("Deinstalling CCMS parameter reporting")


        self.handler.setPattern("^[ \t]*PlugIn.*","# PluginIn") 
        self.handler.purgeFile(agents.sapccmsrIni())


        if switches.unregistFlag:
            agents.unregisterCcms()


        trex.delCcmsFromProfile()


        log.log("CCMS parameter reporting deinstalled.")
    #}  uninstall()
#}  end ManageCCMS








#------------------------------------------------------------------------------
#     ManageLogMonitor: bring Log File Monitor templates to 'logmon' directory
#------------------------------------------------------------------------------
class ManageLogMonitor:
#{ 
    def __init__(self):
    #{
        log.tag("Manage LogMonitor")
        log.debug("initializing")
        self.files = [
            "Daemon_logmon.ini",
            "IndexServer_logmon.ini",
            "NameServer_logmon.ini",
            "Preprocessor_logmon.ini",
            "QueueServer_logmon.ini",
            "WebServer_logmon.ini",
            "RfcServer_logmon.ini" ]
        self.handler = FileHandler()
        rep1 = "DIRECTORY=" + trex.makePath("trace")
        rep2 = 'MONITOR_CONTEXT="TREX on ' + trex.trexHost() + " instance " + trex.trexInstance()+ ' - log files"'
        self.handler.addPattern("^DIRECTORY=.*$",        self.handler.doubleBS(rep1))
        self.handler.addPattern("^MONITOR_CONTEXT=.*$",  self.handler.doubleBS(rep2))
    #}


    def check(self):
        log.debug("switched " + ("off", "on")[switches.logmonFlag])
        return switches.logmonFlag


    def run(self):
        log.log(" ")
        if switches.installing(): self.install()
        else:                     self.uninstall()


    def install(self):
    #{
        #-- add DIR_LOGGING to instance profile, if missing
        agents.addDirLogging()


        #-- create 'logmon' directory
        self.installPath = agents.logmonPath() # 'logmon' subdirectory
        if not os.path.exists(self.installPath):
            try:
                os.makedirs(self.installPath)
            except:
                tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
                for l in tr: log.error(l)
                raise Abort('Cannot create directory "%s"' % self.installPath)




        for file in self.files:
        #{
            inFile = os.path.join(trex.trexExe(), "CCMS", file)
            outfile = os.path.join(self.installPath, trex.trexSid() + "_" + trex.trexInstance() + "_" + trex.baseName() + "_" + file)
            # outfile is like SID_03_P102183_IndexServer_logmon.ini


            self.handler.convertFile(inFile, outfile)
        #}
        log.log("Log monitoring installed.")
    #}




    def uninstall(self):
    #{
        self.installPath = agents.logmonPath() # 'logmon' subdirectory
        for file in self.files:
        #{
            iniFile = os.path.join(self.installPath, trex.trexSid() + "_" + trex.trexInstance() + "_" + trex.baseName() + "_" + file)
            self.handler.removeFile(iniFile)
        #}
        log.log("Log monitoring deinstalled.")
    
    #}
#}  end ManageLogMonitor








#------------------------------------------------------------------------------
#      set up Process Monitoring: bring 'dev_proc' file to saposcol working dir
#------------------------------------------------------------------------------
class ManageProcessMonitor:
#{ 
    def __init__(self):
    #{
        log.tag("Manage ProcessMonitor")
        log.debug("initializing")
        self.handler = FileHandler()
        self.monitorFile = os.path.join(agents.procmonPath(), trex.trexSid() + "_" + trex.trexInstance() + "_procmon.ini")
        self.handler.addPattern(".*TREX.*")
        self.handler.addPattern(".*httpd.*")
        if (os.name == "nt"):
            self.templateFile = os.path.join(trex.trexExe(), "CCMS", "W2K", "dev_proc")  # Windows version is in subdirectory
            self.template = """$PROC
*dllhost.exe*           MTE_CLASS=TREX_PROC
*Inetinfo.exe*          MTE_CLASS=TREX_PROC
*sapccmsr.exe*          MTE_CLASS=TREX_PROC
*saposcol.exe*          MTE_CLASS=TREX_PROC
*python.exe*            MTE_CLASS=TREX_PROC
*TREXDaemon.exe*        MTE_CLASS=TREX_PROC
*TREXNameServer.exe*    MTE_CLASS=TREX_PROC
*TREXIndexServer.exe*   MTE_CLASS=TREX_PROC
*TREXPreprocessor.exe*  MTE_CLASS=TREX_PROC
*TREXQueueServer.exe*   MTE_CLASS=TREX_PROC
*TREXRfcServer.exe*     MTE_CLASS=TREX_PROC
$"""
        else:
            self.templateFile = os.path.join(trex.trexExe(), "CCMS", "dev_proc")
            self.template = """$PROC
*httpd*                 MTE_CLASS=TREX_PROC
*sapccmsr*              MTE_CLASS=TREX_PROC
*saposcol*              MTE_CLASS=TREX_PROC
*python*                MTE_CLASS=TREX_PROC
*trx.sap%s_TRX%s*       MTE_CLASS=TREX_PROC
*TREXNameServer.x*      MTE_CLASS=TREX_PROC
*TREXIndexServer.x*     MTE_CLASS=TREX_PROC
*TREXPreprocessor.x*    MTE_CLASS=TREX_PROC
*TREXQueueServer.x*     MTE_CLASS=TREX_PROC
*TREXRfcServer.x*       MTE_CLASS=TREX_PROC
$""" % ( trex.trexSid(), trex.trexInstance() )
    # *TREXDaemon.x* has been renamed by sapstartsrv to *trx.sapSID_TRX92*
    #}  end __init__


    def check(self):
        log.debug("switched " + ("off", "on")[switches.procmonFlag])
        return switches.procmonFlag


    def run(self):
        log.log(" ")
        if switches.installing(): self.install()
        else:                     self.uninstall()


    def install(self):
    #{
        agents.checkDir(agents.procmonPath()) # mkdirs if not existent
        f = file(self.monitorFile, "w+") # truncate if existent
        f.write(self.template)
        f.close()
        log.log("File created: %s" % self.monitorFile)
#DEL    if (self.handler.existFile(self.monitorFile)):
#DEL        if self.handler.matchFile(self.monitorFile):
#DEL            log.log("Trex Process Monitoring already installed in %s; has not been overwritten." % self.monitorFile);
#DEL            return
#DEL        else:
#DEL            # insert the template entries into it 
#DEL            self.handler.setPattern("^[$]$", self.template)
#DEL            self.handler.convertFile(self.monitorFile)
#DEL            agents.restartSaposcol();
#DEL    else:
#DEL        self.handler.copyFile(self.templateFile, self.monitorFile)
        log.log("Trex Process monitoring installed.")
    #}


    def uninstall(self):
        self.handler.removeFile(self.monitorFile)
        log.log("Trex Process monitoring deinstalled")
#DEL    if (self.handler.matchFile(self.monitorFile)):
#DEL    #{
#DEL        if (agents.getIsLastMonitored()):
#DEL            self.handler.purgeFile(self.monitorFile)
#DEL            log.log("Trex Process monitoring deinstalled")
#DEL            agents.restartSaposcol();
#DEL        else: 
#DEL            log.debug("This is not last Trex instance, Process Monitoring remains installed.")
#DEL    #}
#DEL    else:
#DEL        log.log("Process monitoring was already deinstalled.")
#}  end ManageProcessMonitor












#------------------------------------------------------------------------------
#     install GRMG Heart Beat: bring GRMG.installed.xml to 'GRMG' subdirectory
#------------------------------------------------------------------------------
class ManageGrmg:
#{
    def __init__(self):
    #{
        log.tag("Manage Grmg Heartbeat")
        log.debug("initializing")
        self.httpdUrl = ""
        self.uploader = FileHandler()
        self.uploaderFile = os.path.join(agents.grmgPath(), "GRMG_config." +  trex.trexSid() + "_" + trex.trexInstance() + "." + trex.baseName() + ".xml")
        self.daemonConfig = trex.makePath("TREXDaemon.ini") 
    #}        


    def check(self):
        log.debug("switched " + ("off", "on")[switches.grmgFlag])
        return switches.grmgFlag 


    def run(self):
        log.log(" ")
        if switches.installing(): self.install()
        else:                     self.uninstall()


    def install(self):
        self.daemon = Daemon()
        agents.addDirLogging() #-- add DIR_LOGGING to instance profile, if missing
        self.installConfigFile()
        self.installUploadFile()
        self.activateGrmg()
        log.log("GRMG Heart Beat installed.")


    def httpInstalled(self):
        is_installed = "X"
        not_installed = ""
        c = ConfigParser.ConfigParser()
        c.OPTCRE = re.compile(r'(?P<option>[^:=\s][^:=]*)(\s*(?P<vi>[:=])\s*)?(?P<value>.*)$')
        c.read([os.path.join(trex.trexHome(), "TREXWebServer.ini")])
        try:    url = c.get("HTTPSERVER", "URL")
        except: url = ""
        try:    url_iis = c.get("HTTPSERVER", "URL_IIS")
        except: url_iis = ""
        try:    url_apache = c.get("HTTPSERVER", "URL_APACHE")
        except: url_apache = ""
        if url == "http://XXXhostnameXXX:XXXhttpServerPortXXX/TREXHttpServer/TrexIsapiExt.dll": return not_installed
        if url:                               self.httpdUrl = url;        return is_installed
        if os.name != 'posix' and url_iis   : self.httpdUrl = url_iis;    return is_installed
        if os.name == 'posix' and url_apache and self.daemon.webServerActive():
                                              self.httpdUrl = url_apache; return is_installed
        return not_installed


    def httpUrl(self):
        if self.httpdUrl: return self.httpdUrl.replace("$(SAPLOCALHOST)",trex.trexHost()).replace("$(SAPSYSTEM)",trex.trexInstance())  + "?CMD=PING"
        return ""








    def installConfigFile(self):
    #{
        isPort = self.daemon.indexServerPort() or trex.indexServerPort() # only if first value is 0, take second
        isFlag = (isPort>0) and "X" or ""
        qsPort = self.daemon.queueServerPort() or trex.queueServerPort()
        qsFlag = (qsPort>0) and "X" or ""
        psPort = self.daemon.preproServerPort() or trex.preproServerPort()
        psFlag = (psPort>0) and "X" or ""
        nsPort = self.daemon.nameServerPort() or trex.nameServerPort()
        nsFlag = (nsPort>0) and "X" or ""
        csPort = self.daemon.cruiserServerPort() or trex.cruiserServerPort()
        csFlag = (csPort>0) and "X" or ""


        # formerly # fileName = os.path.join(trex.trexExe(), "python_support", "grmg", "installed.xml")
        fileName = os.path.join(trex.trexHome(), "TREXGrmgServer.xml")
        outFile = open(fileName, "w+")
        configLines = [
            '<?xml version="1.0" encoding="UTF-8"?>' ,
            '<components>' ,
            '   <component name="IS_01" installed="%s">'         % isFlag,
            '       <property name="HOST" value="localhost"/>' ,
            '       <property name="PORT" value="%s"/>'          % isPort,
            '   </component>' ,
            '   <component name="QS_01" installed="%s">'         % qsFlag,
            '       <property name="HOST" value="localhost"/>' ,
            '       <property name="PORT" value="%s"/>'          % qsPort,
            '   </component>' ,
            '   <component name="PP_01" installed="%s">'         % psFlag,
            '       <property name="HOST" value="localhost"/>' ,
            '       <property name="PORT" value="%s"/>'          % psPort,
            '   </component>' ,
            '   <component name="NS_01" installed="%s">'         % nsFlag,
            '       <property name="HOST" value="localhost"/>' ,
            '       <property name="PORT" value="%s"/>'          % nsPort,
            '   </component>' ,
            '   <component name="CS_01" installed="%s">'         % csFlag,
            '       <property name="HOST" value="localhost"/>' ,
            '       <property name="PORT" value="%s"/>'          % csPort,
            '   </component>' ,
            '   <component name="WS_01" installed="%s">'         % self.httpInstalled(),
            '       <property name="URL" value="%s"/>'           % self.httpUrl(),
            '   </component>' ,
            '</components>'
            ]
        for line in configLines:
        #{
            outFile.write(line + '\n')
            log.debug("config line: " + line)
        #}
        outFile.close
        log.log("Created GRMG configuration file: " + fileName)
    #} installConfigFile()






    def scenarios(self):
    #{
        component = """
                                <component>
                                        <compname>%s</compname>
                                        <compversion>01</compversion>
                                        <comptype>Not Used</comptype>
                                        <comptexts>
                                                <comptext>
                                                        <complangu>EN </complangu>
                                                        <compdesc>%s</compdesc>
                                                </comptext>
                                        </comptexts>
                                        <properties>
                                                <property>
                                                        <propname>foo</propname>
                                                        <proptype/>
                                                        <propvalue> bar</propvalue>
                                                </property>
                                        </properties>
                                </component>\n"""
        components = ""
        if self.daemon.indexServerPort()  > -1: components += component % ( "IS_01", "IndexServer" )
        if self.daemon.queueServerPort()  > -1: components += component % ( "QS_01", "QueueServer" )
        if self.daemon.preproServerPort() > -1: components += component % ( "PP_01", "PreProcessor" )
        if self.daemon.nameServerPort()   > -1: components += component % ( "NS_01", "NameServer" )
        if self.daemon.cruiserServerPort()> -1: components += component % ( "CS_01", "Cruiser" )
        if self.daemon.webServerActive()      : components += component % ( "WS_01", "WebServer" )


        grmgDescription = "TREX %s on host %s" % ( trex.trexInstance(), trex.trexHost() )
        grmgServerUrl   = "http://%s:%d/"      % ( trex.trexHost(),     self.grmgPort )


        header = """<?xml version="1.0" encoding="UTF-8"?>
<customizing>
        <control>
                <grmgruns>X</grmgruns>
                <runlog>X</runlog>
                <errorlog>X</errorlog>
        </control>
        <scenarios>
                <scenario>
                        <scenname>TREX</scenname>
                        <scenversion></scenversion>
                        <sceninst>001</sceninst>
                        <scentype>URL</scentype>
                        <scenstarturl>%s</scenstarturl>
                        <scenstartmod>Not Used</scenstartmod>
                        <scentexts>
                                <scentext>
                                        <scenlangu>EN</scenlangu>
                                        <scendesc>%s</scendesc>
                                </scentext>
                        </scentexts>
                        <components>\n""" % ( grmgServerUrl, grmgDescription )
        footer = """
                        </components>
                </scenario>
        </scenarios>
</customizing>"""
        self.xml = header + components + footer
        log.debug("Following text will be content of GRMG upload file " + self.uploaderFile)
        log.debug(self.xml)
    #}  scenarios()










    def installUploadFile(self):
    #{
        
        self.grmgPort = 30000 + (int(trex.trexInstance()) * 100) + 6
        configHtml = os.path.join(trex.trexHost(), "doc", "TrexInstallationSummary.html") 
        while self.portIsInUse(self.grmgPort, configHtml):
        #{
            self.grmgPort += 1
        #}
#        if 0:
#            self.uploader.addPattern("(.*<scendesc>).*(</scendesc>)", r'\1TREX %s on host %s\2' % (trex.trexInstance(), trex.trexHost()) )
#            self.uploader.addPattern("(.*<scenstarturl>).*(</scenstarturl>)", r'\1http://%s:%d/\2' % (trex.trexHost(), self.grmgPort) )
#            templateFile = os.path.join(trex.trexExe(), "python_support", "grmg", "GRMG_config.xml")
#            self.uploader.convertFile(templateFile, self.uploaderFile)
        self.scenarios()
        try:
            agents.checkDir(agents.grmgPath())
            f = open(self.uploaderFile, "w")
            f.write(self.xml)
            f.close()
        except:
            tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
            for l in tr: log.error(l)
            raise Abort("Creation of GRMG upload file %s aborted" % self.uploaderFile)
        log.log("Created GRMG upload file " + self.uploaderFile)


    #}


    def portIsInUse(self, port, inFileName):
        checker=">%d<td>" % port
        fh = FileHandler();
        fh.addPattern(checker)
        return fh.matchFile(inFileName)


    #   activate the GRMG daemon in the TREXDaemon.ini, correct the program start values
    def activateGrmg(self):
    #{
        activator = FileHandler()
        activator.addPattern(r'^programs\s*=[^#\n\r]*grmg')
        if not (activator.matchFile(self.daemonConfig)):
            log.log("Activating GRMG in TREXDaemon.ini")
            activator.setPattern(r'(^programs\s*=[^#\n\r]*)', r'\1,grmg')
            #OUT# program=trex.trexExe() + os.sep + "python_support" + os.sep + "grmg" + os.sep + "TREXgrmg.py"
            #OUT# argline='arguments=%s %d' % (program, self.grmgPort)
            #OUT# startdir=trex.trexExe() + os.sep + "python_support" + os.sep + "grmg"
            #OUT# # here we are ONLY correct, if there are no extra lines with 'grmg': we should have an INI file class!
            #OUT# activator.addPattern('^arguments=.*TREXgrmg.py', activator.doubleBS(argline))
            #OUT# activator.addPattern('^startdir=[^#]*grmg', activator.doubleBS('startdir=' + startdir))
            activator.convertFile(self.daemonConfig)
        if not self.daemon.grmgSection():
            lines = [
            "",
            "[grmg]",
            "executable=python",
            "arguments=TREXgrmg.py",
            "startdir=${DIR_INSTANCE}/exe/python_support/grmg",
            "instances=1",
            "stdout=${SAP_RETRIEVAL_PATH}/trace/grmg.out",
            "stderr=",
            "flags=COMPRESSBACKUP"]
            self.daemon.append(lines)
        if os.name=="posix": os.system('TREXDaemon.x -reconfig')
        else:                os.system('TREXDaemon.exe -reconfig')
    #}  activateGrmg()




    def uninstall(self):
    #{
        log.log("Deinstalling GRMG Heart Beat.")
        # remove the xml file from /usr/sap/tmp/grmg
        self.uploader.removeFile(self.uploaderFile)
        # the files inside trex: installed.xml and the entry in TREXDaemon.ini
        # are to be deleted by SAPINST deinstallator
        self.uploader.setPattern("^([ \t]*programs[ \t]*[=][ \t]*)grmg[ \t]*[,]*(.*)",
                                 r'\1\2') #delete "grmg" directly after 'programs'
        self.uploader.addPattern("^([ \t]*programs[ \t]*=[^#]*)[,]+[ \t]*grmg(.*)",
                                 r'\1\2') #delete ",grmg"
        if self.uploader.matchFile(self.daemonConfig):
            log.log("Deactivating GRMG in TREXDaemon.ini")
            self.uploader.convertFile(self.daemonConfig)
        log.log("GRMG Heart Beat deinstalled.")
    #}
    
#}  end ManageGrmg












#------------------------------------------------------------------------------
#   main program, also used from ccms_deconfig.py
#------------------------------------------------------------------------------
def main():
#{
    result=0


    global switches
    switches = Options()
    switches.check()




    global log


    try:
    #{


        log = Log("main", switches.debugFlag, switches.logfile)
        log.separator()
        if switches.installing():
            log.log("ccms_config.py started: installing supportability features")
        else:
            log.log("ccms_deconfig.py started: uninstalling supportability features")
        
        log.separator()


        global testFAKER
        testFAKER = TestFAKER()


        global trex
        trex = TrexHOME()
        trex.check()


        global agents
        agents = Agents()
        if agents.check():
            agents.run()


        # note: the following classes are dependend from trex,agents,switches,log


        ccms = ManageCCMS()
        if ccms.check():
            ccms.run()


        logmon = ManageLogMonitor()
        if logmon.check():
            logmon.run()


        procmon = ManageProcessMonitor()
        if procmon.check():
            procmon.run()


        grmg = ManageGrmg()
        if grmg.check():
            grmg.run()


        if switches.restartFlag:
            agents.restart()


    #}
    except Skip, s:
        s.logging()
        log.error("Supportability features installation skipped.")
        log.log("Script may be started later by hand") 
        result=0      # report OK, but nothing could be done!
    except Abort, a:
        a.logging()
        result=1
        if switches.forceFlag: result = 0
    except:
        log.error("Caught an unexpected exception, aborted")
        tr = traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback)
        for l in tr: log.error(l)
        result=1
        if switches.forceFlag: result = 0


    log.tag("main")
    log.log("Finishing with result %d" % result)
    log.separator()
    return result
#}  end main


if __name__ == '__main__':
    sys.exit(main())
#])}

Kindly Advise

Regards.

Ahmad Salam

Add comment
10|10000 characters needed characters exceeded

  • Follow
  • Get RSS Feed

2 Answers

  • Jan 17, 2016 at 06:50 PM

    Hi Ahmad.

    1. What is the Trex SP level? Could you share the detail error message ?

    2. Could you refer the SAP Note along with python script

    1978130 - TREX 710: REV 58 . Configuration of the GRMG with the ccms_config.py failed


    Regards

    SS

    Add comment
    10|10000 characters needed characters exceeded

  • Jan 18, 2016 at 10:05 AM


    Hi Ahmad,

    could you please share the error message you get when running the script.What's the shell you are working - BASH,SH,CSH?
    Did you execute the script with TREX-SIDadm user?
    Did you set the TREX environment using the TREXSettings.sh or TREXSettings.csh (dependent on shell) as first? I suppose , it wasn't done and you got something like "ImportError no module fuzzypy".

    Best regards,
    Mikhail

    Add comment
    10|10000 characters needed characters exceeded