Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Deleted user
0 / 0 points
Next:
badges earned

Your Points Tracker
Challenges
Leaderboard
  • Global
  • Feed

Badge for your thoughts?

You're enrolled in our new beta rewards program. Join our group to get the inside scoop and share your feedback.

Join group
Recognition
Give the gift of kudos
You have 0 kudos available to give
Who do you want to recognize?
Why do you want to recognize them?
Kudos
Great job appreciating your peers!
Check back soon to give more kudos.

Past Kudos Given
No kudos given
You haven't given any kudos yet. Share the love above and you'll see it here.

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

How do I read JSON into a Python Class?

I have a JSON configuration file that looks like this:

{
    "container":{
        "image":"reporting_engine",
        "name":"reporting_engine",
        "version":"0.01",
        "network":{
            "host":{
                "ip":"127.0.1.1",
                "port":"5000"
            },
            "container":{
                "ip":"127.0.1.1",
                "port":"5000"
            }
        },
        "storage":{
            "host":"/home/scaldwell/rpe/data",
            "container":"/opt/app/data"
        }
    }
}

I want to read this JSON configuration file into a Python object so I can address properties in an intuitive way programmatically.

I realize I could do something like this: 

import json
with open(config_file,'r') as f:
    config=json.loads(f.read())

and then use my configuration parameters like this:

print config["network"]["host"]["ip"]

 

While that is acceptable Python, it's about as elegant as a Trump speech and possibly just as likely to have errors since there all of those open and closing brackets and quotes.

 

I would rather a solution that allows me to do the same print like this:

print config.network.host.ip

...like a civilized code monkey. wink

1 answer

1 accepted

1 vote
Answer accepted

This code works on my current personal project. It allows me to use config.py to read JSON config files from a shell script that launches and manages my docker containers while also using that same config.py from other python scripts (for the containerized application) to read config properties.

Config files are like Highlander, IMHO. In the end there should be only one!

 

Here is my solution (config.py):

#!/usr/bin/env python
import os
import sys
import time
import json
import inspect
from argparse import ArgumentParser

def parse_args():
    parser=ArgumentParser(
        description="read/write json config file."
    )
    parser.add_argument(
        "--debug",
        action="store_true",
        default=False,
        help="Determines whether verbose log messages should be printed."
    )
    parser.add_argument(
        "--read",
        action="store_true",
        default=False,
        help="Read the file (--config) and return a key (requires --key)."
    )
    parser.add_argument(
        "--write",
        action="store_true",
        default=False,
        help="Write a key (--key) value (--value) pair to --config"
    )
    parser.add_argument(
        "--key",
        dest="key",
        type=str,
        help="Key name to be read/written"
    )
    parser.add_argument(
        "--value",
        dest="value",
        type=str,
        help="Data value to be written."
    )
    parser.add_argument(
        "--config",
        dest="config",
        default="app.json",
        type=str,
        help="The configuration file to be read/written."
    )
    a=parser.parse_args()
    if a.read or a.write:
        if a.read and a.write:
            print "You cannot use both --read and --write concurrently"
            sys.exit(1)
        else:
            return a
    else:   
        print "You must specify either --read or --write"
        sys.exit(1)
class Config:
   
    def __init__(self,oName,is_file=False,i=0,debug_mode=False):
        self.debug_mode=debug_mode
        self.i=i
        self.type=None
        if is_file:
            self.oName="root"
            self.type="root"
        else:
            self.oName=oName
            self.type="child"
        self.debug("init {}[{}] starting".format(self.oName,self.type))
        if is_file:
            j=None
            try:
                with open(oName,'r') as f:
                    j=json.loads(f.read())
                self.debug(
                    "----Raw Input----\n\n{}\n".format(
                        json.dumps(j,indent=4)
                    )
                )
                self.deserialize(j,i+1)
            except Exception as e:
                self.debug("Could not load config file.  Error:{}".format(e))
                sys.exit(1)
            
        self.debug("init {}[{}] complete".format(self.oName,self.type))
    def deserialize(self,j,i=0):
        self.i=i
        try:
            self.debug(" ")
            self.debug("---deserializer loop started")
            self.debug("+>json:{}".format(j))
            for k,v in j.viewitems():
                self.debug("calling walk() with '{}'".format(k))
                self.walk(k,v,i+1)
                self.debug("returned from walk() '{}'".format(k))
            self.debug("---deserializer loop complete.")
        except Exception as e:
            self.debug("Could not deserialize config. Error:".format(e))
            sys.exit(1)
        self.i-=1
    def walk(self,key_name,key_value,i=1):
        self.i=i
        self.debug("walking the config:{}".format(i))
        try:
            o=None
            if type(key_value) is dict:
                o=Config(oName=key_name,i=i)
                o.deserialize(key_value,i+1)
                self.debug("walk(): recovered from recursive call.")
                assert key_name != "oName", "oName is reserved for parser"
                setattr(self,key_name,o)
            else:
                self.debug("Set value:{},{}".format(key_name,key_value))
                assert key_name != "oName", "oName is reserved for parser"
                setattr(self,key_name,key_value)
        except Exception as e:
            self.debug("config.walk() Error:{}".format(e))
            sys.exit(1)
        self.i-=1
    def dump_config(self,i=0,data_only=False):
        old_debug_mode=self.debug_mode
        self.debug_mode=True
        self.i=i
        if i == 0:
            c="*"
        else:
            c="-"
        self.debug("{0}DUMP CONFIG START ({1}:{2}){0}".format(c*3,self.type,self.oName))
        for k in dir(self):
            if data_only and \
               not k.startswith("_") and \
               not callable(getattr(self,k)):
                self.debug("{0:15}:{1}".format(k,getattr(self,k)))
            if isinstance(getattr(self,k),Config):
                self.i+=1
                o=getattr(self,k)
                o.dump_config(self.i+1,data_only)
                self.i-=1
        self.debug("{0}DUMP CONFIG ENDS ({1}:{2}){0}".format(c*3,self.type,self.oName))
        self.debug_mode=old_debug_mode
    def debug(self,m):
        try:
            if self.debug_mode:
                print "{0:15}|{1:<2}| {2} [{3}]:{4}".format(
                    int(time.time()*10000000000),
                    hex(self.i),
                    " . "*self.i,
                    self.oName,
                    m
                )
        except Exception as e:
            print "Could not print message: Error:{}".format(e)
            sys.exit(1)
    def getChildAttribute(self,index,object,path):
        self.i+=1
        assert isinstance(self,Config),"self must be <instance>"
        assert type(index) is int, "index must be <int>"
        assert isinstance(object,Config),"object must be <instance>"
        assert type(path) is list, "path must be <list>"
        for i in path: 
            assert type(i) in [str,unicode], "path elements must be <str>"
        assert index >= 0, "index must be > 0"
        assert len(path) > 0, "len(path <list>) must be > 0"
        self.debug("position[{}]:{}".format(index,path[index]))
        o=getattr(object,path[index])
        if (index < len(path)) and isinstance(o,Config):
            return self.getChildAttribute(index+1,o,path)
        else:
            return o
        self.i-=1
    def read_value(self,k):
        if self.debug_mode:self.dump_config(data_only=True)
        self.i=0
        try:
            path=k.split('.')
            self.debug("path: {}".format(path))
            o=getattr(self,path[0])
            self.debug("o:{}:{}".format(o,o.oName))
            return 0, self.getChildAttribute(
                                    index=1,
                                    object=o,
                                    path=path
            )
        except Exception as e:
            return 1,e
    def write_value(self,k,v):
        try:
            assert k is not None, "Expected a key string (--key)"
            assert v is not None, "Expected a value string (--value)"
            assert type(k) is str, "--key must specify a string."
            assert type(v) is str, "--value must specify a string."
        except Exception as e:
            print "Error: {}".format(e) 
            sys.exit(1)
        try:
            print "write_value() not implemented yet."
        except Exception as e:
            print "write_value() Error: {}".format(e)
            sys.exit(1)
def main():
    args=parse_args()
    config=Config(args.config,is_file=True,debug_mode=args.debug)
    if args.read:
        e,v=config.read_value(args.key)
        if args.debug:
            print "Key:'{}', Value:'{}', ExitCode:{}".format(args.key,v,e)
        else:
            print v
        sys.exit(e)
    elif args.write:
        sys.exit(config.write_value(args.key,args.value))
    else:
        print "Uncaught input parameter error.  Expected --read or --write"
        sys.exit(1)
if __name__ == "__main__":
    main()

This allows me to have a config file like this:

{
    "container":{
        "image":"reporting_engine",
        "name":"reporting_engine",
        "ip":"127.0.1.1",
        "port":"5000",
        "volume":"/storage"
    },
    "app":{
    }
}

..and to then use the utility to fetch JSON config elements like this: 

python common/config --read --config reporting_engine/app.json --key container.image

Which will write the following to stdout: 

reporting_engine

As the JSON config changes, the code doesn't.  That's my favorite kind of software (the flexible kind). wink

Steven Behnke Community Leader Aug 13, 2016

image2016-8-13 11:57:48.png

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Asked in Jira Service Management

JSM June ask me anything (AMA)

Hello Community members! We’re wrapping up the end of JSM June with an Ask Me Anything (AMA) with the Jira Service Management product team. This is your chance to ask all your ITSM questions to o...

233 views 12 14
View question

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you