Home >
>
Newbie Follow-Alongs
Django Unchained - Old Guy’s Scary Mission (42)
05-19-2022 04:12 PM
#1
larsometer (Senior Member)
Django Unchained - Old Guy’s Scary Mission
So here it is, my new follow along.
This time I chose a mission that scares the shit out of me.
There is this project I am dreaming about for several months now. However so far I have found all sorts of excuses. In other words, I chickened out.
Every day I read data science articles…. I don’t know why but I am kinda addicted to that stuff. Before I recognise it I get wet eyes and lose myself into daydreaming.
After a while I can hear a soft voice speaking in my head:
I want to create something like this, I want it so badly.
All the memories of not having started my project suddenly come back and make me feel miserable.
This has to stop!
Fighting dreams doesn’t work. Committing to realising dreams tho can work very well. And this is what I want to achieve with this follow along.
What’s my mission?
I want to create a Django application to make my life and hopefully that of many others a bit easier.
First step will be to have analytics for TC IVR. Basically it will be a tool that helps to find the right geos and the best timing for running IVR camps.
The following screenshot from a flask plotly dash mockup gives you an idea what the result could look like.
Once I have mastered this I will think about the next steps.

Why is it scary?
Django is a huge framework that is far beyond my current skillset.
By far I mean so far away that just watching tutorials intimidate me.
Also I basically have no time for such a project yet but now have committed to it.
This means I will need to change and optimise many things in my daily life to ensure a 2-3h/day time slot for Django.
Also I will have less time to watch my tracker. Good thing tho is that this project helps to overcome this addiction.
Why have I chosen Django?
Simple reason is that I want to have something stable and secure that also comes with all sorts of well established addons.
And… it needed to be Python since this is the language I most familiar with.
If it were only for my purposes I most likely would have been better off with a micro framework.
Why I want to do it all by myself?
Sure, I could just hire some developers and do the job for me. But that wouldn’t be any fun.
Part of me wants to prove to himself that he can do it. Other parts just want to get some traction into machine based problem solving and shaping my brain for it (neuroplasticity… you know).
What will be the next steps?
From tomorrow on I will invest at least 2hrs/day to understand Djangos structure.
After that I will do the typical hello world thingy to get a first real feeling.
Then I will make a plan on how to want to structure my project.
So don’t expect daily updates from me. I rather will do on a milestone basis.
Nevertheless I would be happy for a reminder in case I haven’t posted for a longer time. 
05-19-2022 06:22 PM
#2
craigm (Veteran Member)
I'm fairly familiar with Django.
Feel free to post Qs here, see if I can help.
05-22-2022 02:36 PM
#3
larsometer (Senior Member)
First update... 
So far I was a guy who first wanted to get some general understanding of a topic before going into action. Well this approach doesn't seem to be the best for Django. It is far too complex, tho everything that was explained in the videos I watched seemed quite logical to me. It felt a bit like connecting the dots... not just a few but many.
Instead of getting overwhelmed I decided to break things down and solve things step by step.
Cool thing is that there seems to be a tuturial for every single thing you can do with Django. So I can follow my philosophy of rip 'n adjust. 
Today I managed to get my first site run from a local server, basically the hello world thingy. Took me around 10minutes because of:
- find YT tutorial from someone who doesn't have a hot potatoe in his mouth while speaking 
- create several directories / structures --> can be done with python commands
- create several files and x-reference them:
- in the project file you have to define that there is a /home directory for the site
- in the app file you define that you have an index file
- the above mentioned files are both called "urls.py" but are in different directories --> as a noob you can easily confuse which is which
- write view.py file with the actual "Hello World" code.
- correcting some errors --> BUT Django error messages seem to be amazingly clear / understandable.


After having achieved my first "milestone" I will continue with the following points:
- create simple homepage with a login button
- make sure that page with diagram can only accessed when logged in
- create a site which contains an plotly dash diagram (like the one from my first post but without dropdown)
- make diagram interactive with dropdown menus
--> Focus is on having the functions working in local environment, no focus on look nice (will follow later)
Not sure how long I will need for all this but my current estimation is at least a week.
To be continued...
05-22-2022 07:31 PM
#4
Rhino (Senior Member)
Just curious to know, how old are you?
05-23-2022 05:26 AM
#5
larsometer (Senior Member)
I am 46 years old, but sometimes I feel like 47 
05-23-2022 08:34 AM
#6
matuloo (Legendary Moderator)

Originally Posted by
larsometer
I am 46 years old, but sometimes I feel like 47

I know that feeling! I'm 44 but already feel like 45 hehe
05-23-2022 03:22 PM
#7
craigm (Veteran Member)
You should be reading 'Two scoops of Django' if you haven't yet, it'll help you understand best practices with Django.
Some of the early chapters will help you set up virtual environments and virtualenvwrapper which I highly recommend you do.
You should also scaffold your project with django cookiecutter rather than with the startproject management command, it'll give you a much better starting point towards a production ready app (it's also covered in that book).
https://github.com/cookiecutter/cookiecutter-django
05-23-2022 04:51 PM
#8
larsometer (Senior Member)
Cool, thank you 
Just have seen that Daniel Roy Greenfeld also has a YT channel where he explains things.
Am not so sure about reading the book. I probably would love it too much. Then I would study it for weeks and do nothing like I did with a 1200 page Python book I once bought.
Virtual Env. I already do with Pycharm. Not so sure what the wrapper thingy does but my Django project is completely isolated from all the rest I do. Guess will understand the wrapper thingy in a few days as well.
05-24-2022 12:31 AM
#9
vortex (Senior Moderator)
@larsometer This is huge! I'm guessing that analyzing the stats is just the first step? Second step being to automatically tweak your campaigns to target the profitable geos during profitable hours?
This is really exciting! Looking forward to seeing results when you have them!
I'm 46 years old this year as well. Kudos to proving to the young'uns that we "old dogs" can learn new tricks still! 
Oh and I guess you can replace the "Soon to come..." in your signature now.
Amy
05-24-2022 08:13 AM
#10
craigm (Veteran Member)

Originally Posted by
larsometer
Virtual Env. I already do with Pycharm. Not so sure what the wrapper thingy does but my Django project is completely isolated from all the rest I do. Guess will understand the wrapper thingy in a few days as well.
It makes it easier to use your virtual environment when using a separate terminal window outside of Pycharm, I don't think you'll need it if you play on working entirely within your IDE.
05-24-2022 09:18 AM
#11
larsometer (Senior Member)
Yes and that is why it scares me so much.
I'm guessing that analyzing the stats is just the first step? Second step being to automatically tweak your campaigns to target the profitable geos during profitable hours?
Actually that is second and third step.
First steps will be to have a website with user access and db-connected diagrams on it.
Then I will need to do a lot of data cleaning since the raw data has quite some outliers and gaps.
After that I want to make it available to like minded people so that they can play with it and can give me feedback on what kind of analytics / indicators they wish for. Of course I have some own ideas but I think that it will help a lot to have many eyes on it.
When this is all done I may do an API for it. So people can set trigger points and also use it for their camps.
So yes... the "end goal" is to move towards automation.
I'm 46 years old this year as well. Kudos to proving to the young'uns that we "old dogs" can learn new tricks still!
Kinda funny... in this FA the STM world learns who is in the 40+ club
Good thing about age tho is that I get tired quickly when I do too many stupid things. However when I do "smart" things I am still excited like an 7 year old.
Oh and I guess you can replace the "Soon to come..." in your signature now.
Done
√
It makes it easier to use your virtual environment when using a separate terminal window outside of Pycharm, I don't think you'll need it if you play on working entirely within your IDE.
Aaah I guess now I understand a little better. So far doing everything in pycharm since it also supports remote deployment.
05-24-2022 01:03 PM
#12
twinaxe (Senior Moderator)
Interesting project my friend.
Funny about it is that I also want to create my first dashboard and I also want to start with the TC IVR API numbers.
I thought for some time what to use for it and i decided against Django and will go with FastAPI, MongoDB and React.
Maybe you can change my mind when I see that you´re making good progress with your Django aproach while my approach won´t work that well.
Or maybe it´s the other way round 
05-26-2022 09:11 AM
#13
diplomat (Member)
You guys may wish to check out Grafana for dashboards. Also, for time-series databases, you may wish to check out Clickhouse database (Postgres works too or Timescale). MongoDB and similar are good until they are not. There's a tool for everything and for dashboards MongoDB is definitely quite bad choice. It's a document database.
05-26-2022 09:41 AM
#14
twinaxe (Senior Moderator)
@diplomat Thanks for your input.
Will check these things out.
05-28-2022 01:58 PM
#15
twinaxe (Senior Moderator)
@diplomat I had a look at Grafana and Clickhouse.
Grafana looks absolutely amazing and I love how you can just add new panels there, it really gets me thinking about it.
So far I decided to go with Falcon.
It has everything I need with it´s charts and tables and so on.
But I will definitely keep Grafana in mind because it looks really nice.
Clickhouse sounds also very interesting but I need to learn so many new things anyway because I am no professional coder at all and now I know already a bit about Mongo so I will stick with it for now.
From what I read it really seems that Clickhouse outperforms many/most other databases but there are also cases where other databases would be better than Clickhouse.
This would mean the perfect setup would be to use different databases for different purposes but the additional effort to also learn these things isn´t worth it for my projects.
In the end the difference in performance will become more apparent when you run at higher loads.
As I said I am no pro coder at all and I am 100% confident that Clickhouse could improve the performance significantly if used properly but I want to build a dashboard basically just for me and for my own campaigns, I guess for such purposes my Fast API + MongoDB + React stack should do a good job.
Probably I would see it different when it´s a commercial product or when it´s planned to run a platform on high volume but I would lose interest when I first need to learn too many new things only for a performance improvement, I want to get things done fast so my main focus is on getting my stuff running first.
But same as with Grafan I will bookmark Clickhouse, even if I don´t use it now it could be helpful later.
They both seem to be great products but it would mean additional affort and when you have to learn basically everything you need to know to build such a platform you´re happy when you keep the stuff as simple as possible 
05-28-2022 02:41 PM
#16
diplomat (Member)
Well, it all depends on what you need to do. I still can't recommend MongoDB for that kind of purpose. Instead, I'd go with PostgreSQL or MySQL (MariaDB).
Clickhouse is excellent for time series and analytics. We use it in our system to store and analyze website traffic that our customers receive. Our Clickhouse setup stores tens of billions of rows of data and it's super fast with its queries. It has tons of incredible functions to use which is why I recommend it for analytics.
Grafana is really nice tool to build dashboards, but it can be too limited if you want to build more in-depth stuff.
Django has incredible ORM support for Postgres and MySQL + tons more. So you are able to build crazy queries without ever typing any SQL. We use Django and Flask heavily in our tech stack because both are incredibly easy to use and super powerful thanks to Python. I personally dislike React (or Javascript in general), but all of our dashboards/panels are written using it... mostly by me, so yeah, lol. Python is incredible tho.. I used to hate it when I thought PHP was amazing, but once I tried it I haven't turned back.
These days there are tons of different technologies for you to use and that's great and confusing at the same time.
05-29-2022 04:02 AM
#17
jaybot (Veteran Member)
Everyone hates on JS 
It has curly braces, so I can't bring myself to like Python more.
Don't like React tho.
05-29-2022 10:58 AM
#18
diplomat (Member)
I personally like React a lot. It's sweet.. much more reasonable than Angular. I do like Vue a bit. We use it for simple components. I don't like JS in the server because it feels like it's bastardized.
This is totally true:

Python then again is a 31 year old language that's still growing in popularity. Yes, it's a bit strange.. no curly braces and tabs for days, not statically typed.. but it's simple and clean and crazy powerful and flexible. Yes, I'm fully aware I sound like this guy:

What I usually suggest people do is just try Python. Give it a chance and then you see if it works for you or not. In life, we have all kinds of prejudices that we learn when browsing the web and hearing all kinds of info from others, and this all develops our opinions, but I say.. have your own opinion. Don't let others design your opinions. Just because it looks different doesn't mean it's bad 
05-29-2022 04:53 PM
#19
jaybot (Veteran Member)

Originally Posted by
diplomat
Lmao. More memes like this will make me switch to Python full-time. I'm a simple man and have simple needs. Spent a lot of time in gamedev where most things were in C and random scripting languages which make heavy use of curly braces. That's literally the only reason. And I know it's stupid.
I like Python a lot. I love reading Python, it's so clean. Cut my teeth on it many years ago. Was so much fun figuring out how to brute force a bunch of the euler problems in one line. I also love many of the similar ideas in Ruby (haven't touched rails).
The only thing I don't like about JS is the asynchronous stuff. I'm really not a web guy, so the whole promise and key stuff makes my brain hurt. So seriously, don't listen to me on programming stuff! Always ask @
diplomat and @
jeremie first, they're the real deal
I'm just here to shitpost
05-29-2022 07:48 PM
#20
larsometer (Senior Member)
2nd update for my follow along.
But... before I start a big thanks to all the contributers here. Am really surprised by the discussions that are going on here. First I thought that programming stuff wouldn't get much attention. Now I am happy to having been wrong.
Maybe you can change my mind when I see that you´re making good progress with your Django aproach while my approach won´t work that well.
@
twinaxe Many roads lead to Rome... they say. For me Django seemed to be a bit more strict on how to do things. So I will follow its patterns and hope that my code does not end up like a big chaos again
@
diplomat Grafana is dope! I played with the demo site and felt like a comander on a space ship exploring infinite spaces.
For the moment it is far too complex for my purposes. Still have to explore which KPIs I will need and how to compute them. Plotly dash seemed to be most inuitive for me.
Currently I use MongoDB for storing the JSON data I retrieve from traffic company. Main advantage is that I don't have to worry when they add fields. Also using a composite primary key was easiest option (for me) to filter out duplicates right from the beginning.
For "production" however I will go with an SQL database since I want to have something stable, reliable and a schema that I understand months later. A time series db wouldn't make sense for my case. Actually I might even use Redis or something of the like in case I really need speed. Data volume wise it is not that much.
@
jason Django also has curly braces... actually quite many for all the tokens that you want to use.
My fingers are already twisted since I dont have these cute braces on my German QWERTZ Mac keyboard. Can reach them with using alt-key but that feels really akward.
Maybe I just should buy a cheap qwerty keyboard.
Now to the actual update...
Watched several YT videos and found out that some explain you the quick and dirty way while others explain how to do things properly. I liked the second approach much more.
Corey Schafer style I like most:
https://www.youtube.com/c/Coreyms
Things slowly begin to make sense. Emphasis is on
slowly.
I thought that setting up a static HTML page would be an easy job. My assumption was that it shouldn't be too much different from ripping and cleaning landers.
Well my experienced helped me quite a bit. Nevertheless I am still impressed how clunky a simple template page can be.
Next job was to get the static files organized in Django. I thought it would be require copying into the right directory... but you additionally need a load static token for each single file in your index html.
Once I undertood, it wasn't all that difficult
Anyways... so here is the result of this week's activities:
[img]https://i.imgur.com/XvSp98x.png[/img

Next steps will be to understand and integrate user management. So basically I want to make sure that users have to log in before they can reach the dashboard page.
05-30-2022 11:53 AM
#21
twinaxe (Senior Moderator)
Well, it all depends on what you need to do. I still can't recommend MongoDB for that kind of purpose. Instead, I'd go with PostgreSQL or MySQL (MariaDB).
Yup, there are specific solutions for everything that are performing better for what they´re specialized in but my platform shall not only consist of a dashboard so in the end I just want a good multipurpose solution.
Our Clickhouse setup stores
tens of billions of rows of data and it's super fast with its queries
There it probably makes much sense but as I said, I want to build a platform just for me with way less data, there it would be kind of overkill.
I don´t know much about Python, Databaes and whatever anyway so for me most important is to get things done instead of getting things perfect from the beginning.
Learning all the additional new things would just block me in the progress, even when it´s a "better" solution.
When I see that the projects scale to dimensions where specialized solutions make more sense I can still shift specific tasks there.
Grafana is really nice tool to build dashboards, but it can be too limited if you want to build more in-depth stuff.
I checked Grafana again and watched some Youtube clips and read about it and it looks absolutely amazing so that I seriously considered using it.
One big disadvantage is that it can only read data as far as I understand but I also need to insert data to the DB or to Python scripts.
These days there are tons of different technologies for you to use and that's great and confusing at the same time.
That´s really a huge issue.
There is just
something for
everything so that you can easily lose the overview.
Also a reason why I try to get my stuff first done with a good multipurpose stack, I can still improve here and there once the foundation is running.

Originally Posted by
diplomat
I personally dislike React

Originally Posted by
diplomat
I personally like React a lot.
Huh?
I'm just here to shitpost
We all enjoy your posts, we probably even love them but we´re also fully aware that you´re just here to shitpost
@twinaxe Many roads lead to Rome
That´s true
For me Django seemed to be a bit more strict on how to do things.
Exactly this was why I decided against Django after I thought about it very long
Now to the actual update...
Looks good dude and step by step you come closer to your goal
05-30-2022 12:01 PM
#22
diplomat (Member)

Originally Posted by
twinaxe
Huh?

Oh damn, I slipped lol. What I meant was I dislike React because it's Javascript, but I like React when I have to build something in Javascript (like dashboards, panels, and stuff). Also, I don't believe in JS in the server-side context. Just because you can do it doesn't mean you should.
It's reasonable to use what you already know. Build a proof of concept however you like and then switch to what really needs to be used. Eventually, you will figure out what kind of limitations MongoDB has (especially when you start doing joins) and then you realize you probably need a relational database and then you will need to migrate your stuff, and then it's totally okay to cry a bit.
05-30-2022 12:12 PM
#23
twinaxe (Senior Moderator)
It's reasonable to use what you already know. Build a proof of concept however you like and then switch to what really needs to be used.
That´s the plan.
No need for additional effort and resources when I don´t even know if I will really need it.
Eventually, you will figure out what kind of limitations MongoDB has
I know man, I know.
But MongoDB is so easy to use and I love its flexibility, that´s why it´s my weapon of choice in the beginning because it helps me alot to get things done faster.
and then it's totally okay to cry a bit.
05-31-2022 05:09 AM
#24
quintyfresh (Senior Member)
@larsometer
I will be watching closely here to see how you do. Good progress so far. As someone who has gotten lost in many languages and frameworks don't worry about learning speed. Things will start to become second nature after some time. About this...
So I will follow its patterns and hope that my code does not end up like a big chaos again
Make sure you comment. And then comment on those comments. And then make footnotes about those comments. And then make comment blocks about those comments. See the point? I have programs from 2014-15 that I only understand today because I commented everywhere and in block a patterns like...
(this is in vb.net cod fyi)
'===============================
'THIS SECTION HANDLES DOING XYX
'CONNECTS TO BLAH BLAH BLAH ON LINE BLAH
'===============================
Whatever subs and functions do things in here
'===============================
'END BLAH BLAH SECTION
'===============================
Is this overkill. Hell yeah. Does it remind me what I did? YES. I also know this is going to sounds really obvious so sorry for those of you who do this already...Version control is an awesome thing, usually with GitHub. It's easy to get a client for it, and every time you finish editing in NEW code on the project commit it to the GitHub. And even keep it private. This way you can track all the historical changes you made and also get old versions etc if you need to. I know this information isn't Django specific so sorry about that. This is more general just keeping track of what you are doing info.
What exactly is this program intended to do? Track hour by hour across all GEOs and track which hours and GEOs seem to perform best? Just want to make sure I am following since I have a TC account but I still don't use it super often.
05-31-2022 10:18 AM
#25
craigm (Veteran Member)

Originally Posted by
larsometer
Currently I use MongoDB for storing the JSON data I retrieve from traffic company. Main advantage is that I don't have to worry when they add fields. Also using a composite primary key was easiest option (for me) to filter out duplicates right from the beginning.
For "production" however I will go with an SQL database
You should just stick with Postgres now then, you don't want to have to deal with that technical debt down the road.
If you're worried about a changing schema like that you can just use Django's
JsonField
05-31-2022 03:35 PM
#26
twinaxe (Senior Moderator)
I hate it when people like @diplomat or @craigm enter a coding discussion.
They have more knowledge than me, they tell how to improve the programming, then I do some research and have to admit that they are somewhat right.
Then all my thoroughly planned steps are flushed down the drain because I want to overthink everything again 
I agree that it´s probably a good idea to switch the database sooner or later but at least in the early stage I will stick with MongoDB.
In my last project I changed the DB structure so often that Mongo with its flexibility was a huge timesaver there.
Will also use it for the new project for now but once the platform is ready in a way that there are no big changes to be expected anymore in the structure I will consider to use another DB if I see performance problems with Mongo 
05-31-2022 03:42 PM
#27
diplomat (Member)

Originally Posted by
twinaxe
I hate it when people like @
diplomat or @
craigm enter a coding discussion.
They have more knowledge than me, they tell how to improve the programming, then I do some research and have to admit that they are somewhat right.
Then all my thoroughly planned steps are flushed down the drain because I want to overthink everything
again
I agree that it´s probably a good idea to switch the database sooner or later but at least in the early stage I will stick with MongoDB.
In my last project I changed the DB structure so often that Mongo with its flexibility was a huge timesaver there.
Will also use it for the new project for now but once the platform is ready in a way that there are no big changes to be expected anymore in the structure I will consider to use another DB
if I see performance problems with Mongo 
The thing is.. MongoDB and Postgres/MySQL are built differently.. if you decide you need to change your database, you have to change your code, queries, and database tables architecture as well. Once that's done, you need to migrate your existing data so that it fits your new database tables. So, if I'm completely honest.. it will be A LOT easier to use the proper database from the beginning and learn how to work with it. Saves you a lot more time and nerves in the future.
The issue is not with performance.. the problem is the structure, and changing the structure of your database can be really difficult.
05-31-2022 05:13 PM
#28
twinaxe (Senior Moderator)
@diplomat Could you explain a in 2-3 easy understandable sentences why exactly Postgres is better than Mongo?
As far as I understand one big difference is that Mongo is doing a good job when you just read/write/edit data in the documents itself without the different documents being connected/linked with each other.
Basically each document is like a container that stores data without being connected to the other containers with data.
When data from different documents is connected/linked in some way it would be better to use a relational database.
Is this correct so far?
And what are situations where MongoDB would be the better/a good choice in your opinion?
For example let´s say I want to create a database for a comic book collection with Publisher, Title and Issue like
Code:
Marvel
Spiderman
Issue 1
Issue 2
Issue 3
Iron Man
Issue 1
Issue 2
Issue 3
Captain America
Issue 1
Issue 2
Issue 3
DC Comics
Superman
Issue 1
Issue 2
Issue 3
Batman
Issue 1
Issue 2
Issue 3
Aquaman
Issue 1
Issue 2
Issue 3
Image Comics
Spawn
Issue 1
Issue 2
Issue 3
Wildstorm
Issue 1
Issue 2
Issue 3
Youngblood
Issue 1
Issue 2
Issue 3
For such a project MongoDB would be a good choice.
When I now add some functions where users can comment or rate the issues it would be better to use Postgres (or another relational DB).
Is this correct so far?
@
larsometer Sorry for spamming your thread
05-31-2022 05:30 PM
#29
larsometer (Senior Member)
Sorry for spamming your thread
No need for sorry... I am really happy that this thread gets so much attention.
And for your comic project.... that is a very typical use case of an SQL/relational DB (with and without comments).
05-31-2022 06:26 PM
#30
diplomat (Member)

Originally Posted by
twinaxe
@
diplomat Could you explain a in 2-3 easy understandable sentences why exactly Postgres is better than Mongo?
Postgres does not have to be BETTER than MongoDB. They are different systems for different requirements. In some cases, Postgres is more convenient than MongoDB.. and in some cases MongoDB may be a better choice. Which one to pick depends on the use case. Postgres is a very capable database server that can do pretty much what MongoDB can do, but the MongoDB cannot do all things what Postgres can do (relations can be tricky, ACID, joins).

Originally Posted by
twinaxe
As far as I understand one big difference is that Mongo is doing a good job when you just read/write/edit data in the documents itself without the different documents being connected/linked with each other.
Yes, and if in the future you need to update something.. that is included in multiple documents then you need to actually update multiple documents. With a SQL database you just replace it in one table/row and voila, that's all. If we are looking at large databases.. you can imagine how much duplicate data you may have. Also querying relational stuff is really expensive. Thanks to JOINs in the SQL database it's really nice and fast and convenient (if you have indexes set up properly).

Originally Posted by
twinaxe
Basically each document is like a container that stores data without being connected to the other containers with data.
When data from different documents is connected/linked in some way it would be better to use a relational database.
Is this correct so far?
Yes, that's correct.

Originally Posted by
twinaxe
And what are situations where MongoDB would be the better/a good choice in your opinion?
MongoDB would work well in a situation where you don't need to have relations in data or have unstructured data. So.. we are talking about logs or storing application specific data like products, user comments and all kinds of metadata and of course all kinds of documents. I guess you could even use it for data archiving if you normalize your relational data.

Originally Posted by
twinaxe
For example let´s say I want to create a database for a comic book collection with Publisher, Title and Issue like
Code:
Marvel
Spiderman
Issue 1
Issue 2
Issue 3
Iron Man
Issue 1
Issue 2
Issue 3
Captain America
Issue 1
Issue 2
Issue 3
DC Comics
Superman
Issue 1
Issue 2
Issue 3
Batman
Issue 1
Issue 2
Issue 3
Aquaman
Issue 1
Issue 2
Issue 3
Image Comics
Spawn
Issue 1
Issue 2
Issue 3
Wildstorm
Issue 1
Issue 2
Issue 3
Youngblood
Issue 1
Issue 2
Issue 3
For such a project MongoDB would be a good choice.
When I now add some functions where users can comment or rate the issues it would be better to use Postgres (or another relational DB).
It's difficult to say if MongoDB is a good or bad choice for that because there's not enough information. Since the structure is really basic and if nothing changes there then sure, I guess it would work just fine.
What I would do instead is create 3 tables: publisher, hero (or whatever, I don't know comics that well), issue.
That kind of setup means that you won't have weird duplicated data issues (if you knowingly don't add them). If something needs to be changed then it can be done easily by updating a row. If you want to be really fancy, you can add another table for hero + issue so you can have multiple heroes associated with a single issue:
That kind of structure allows you to build all kinds of interesting queries without multiple queries or duplicate content.
Anyway, I know this is not the super best example and may not address your questions as you would like, but that's because there's not enough information about your requirements. My experience tells me to use a SQL database because most of the stuff I build is using relational data and for unstructured data I can use JSON field in Postgres (you can even perform queries on that data).
So yeah.. I do believe I could provide more info when I'd have more info as well.
But please keep this in mind. MongoDB is NOT a bad database and Postgres/MySQL are not ultimate AIO databases. They all have a role in the ecosystem.. they just have different ways of operating.
My go to database is MySQL (MariaDB). For fancier stuff I use Postgres.
05-31-2022 07:27 PM
#31
twinaxe (Senior Moderator)
Anyway, I know this is not the super best example and may not address your questions as you would like, but that's because there's not enough information about your requirements.
Oh, don´t get me wrong.
That wasn´t a real life project, it´s just a rough example that came to my mind
I don't know comics that well
Same here, was just the first thing I thought of that could be good to use as an example
But please keep this in mind. MongoDB is NOT a bad database and Postgres/MySQL are not ultimate AIO databases. They all have a role in the ecosystem.. they just have different ways of operating.
Seriously, this is a real challenge to know what´s best to use for a project.
It´s blessing or curse that there are so many different options to choose from.
You can decide between different backends, frontends, databases and whatever.
It´s great to have several options to choose from that are doing specific tasks much better than others but it´s also very tiresome at the same time.
Every time you think you know what to use you find other options that seem to be better.
That´s also why I came up with the Fast API + MongoDB + React stack because I thought it´s a good and solid allrounder that can be used for many different projects.
To keep things as simple as possible I tried to have a good general setup that just gets the job done reliable even when there are other backends/frontends/databases that are superior for this or that but where more additional effort is required then.
Although I see the advantages of using a relational database over MongoDB there´s that one thing that makes Mongo superior for me: That I can change the whole structure basically on the fly or whenever it´s needed.
In my last project this alone saved me so much trouble.
I already planned what I want to do pretty good but while I was working/coding on it I had so many new ideas or approaches where Mongo allowed me to basically rewrite the whole DB structure whenever I wanted (and it happened many times).
Often I think that in many of these decisions you basically have two options.
You can have a solution that´s rock solid and reliable for what it does but where you´re limited in your decisions.
On the other hand you can have a solution that´s maybe not that high performing or "
professional" or whatever you want to call it but it gives you maximum freedom of choice.
Basically kind of
professionalism vs
creativity.
Similar situation with the big brainstorming if it´s better to use Django or Flask.
In the end I decided to go with Flask because I prefer flexibility over proven and solid but limiting.
And then I found Fast API...
Probably a never-ending story with all these things
06-01-2022 08:41 AM
#32
diplomat (Member)
Unfortunately, I cannot agree with you, but it sounds like you have made up your mind so I will not try to convince you to do something else.
But just for your information.. Django can be as flexible as Flask it just carries more weight in features. Another awesome thing about Django is that it literally addresses your concern about rebuilding database structure. The ORM is there to help you do just that. It will manage everything for you and all you need to do is create/edit models. You don't need to touch the database, edit tables manually or anything like this
Also thanks to the ORM, you have a fantastic CRUD admin panel automatically built for you for easier management.
06-01-2022 09:38 AM
#33
jeremie (Moderator)

Originally Posted by
larsometer
Next steps will be to understand and integrate user management. So basically I want to make sure that users have to log in before they can reach the dashboard page.
I would not start with that personally, assuming that you are going to use it alone for now. You can probably use server authentication like the HTAccess files on Apache servers. It takes 5 minutes to implement and you are done.
Rather than spending time on permissions, user access rights... Focus on the core of the system and make it works. Once you scale, you will have money to pay someone to refactor it in a v2 with all the bells and whistles. And probably the user access structure you are thinking of now, will be different once you scale.
06-01-2022 11:59 AM
#34
twinaxe (Senior Moderator)
Unfortunately, I cannot agree with you, but it sounds like you have made up your mind so I will not try to convince you to do something else.
@
diplomat In
THIS THREAD you can read a bit more about the reasons why I decided to go with Fast API + MongoDB + React.
And you can trust me when I say that the decision wasn´t easy and even now I am not sure if it´s the right choice.
I was reading lots of articles, comparisons and whatever to get a better idea what I want to use, at one point I just had to decide because otherwise it probably would never find a end
There were two sentences that I read in these articles that played a big role in my decision against Django:
The first sentence was: "With Flask you build your framework around your scripts, with Django you build your scripts around the framework."
The second sentence that I have read was: "With Flask you learn to code Python, with Django you learn to code Django".
In the last days however I re-thought lots of things and you really got me thinking with the Mongo vs Non-Mongo database stuff.
Should I really want to use another database like MySQL or Postgres then it also would make more sense to use Django because of its ORM.
It´s a dilemma

Originally Posted by
jeremie
You can probably use server authentication like the HTAccess files on Apache servers. It takes 5 minutes to implement and you are done.
That´s what I also thought some time ago for my platform.
Especially as long as I use it only myself this would be the fastest and easiest solution to implement some kind of basic authentification.
06-01-2022 12:33 PM
#35
diplomat (Member)

Originally Posted by
twinaxe
The first sentence was: "With Flask you build your framework around your scripts, with Django you build your scripts around the framework."
The second sentence that I have read was: "With Flask you learn to code Python, with Django you learn to code Django".
Both sentences don't make much sense.
Flask and Django both are frameworks themselves. With both, you write Python not some sub-language. Also "to code Django" doesn't make sense. Django does not have its own syntax or anything like this. For database stuff, you use Django ORM (but can use raw queries too if you are brave enough). The ORM may be tricky to understand in the beginning, but it's absolutely incredible and one of the main features of why I love Django. With Flask, you can use whatever you wish, but Sqlalchemy is a great solution for an ORM. Flask uses Jinja for templates, Django uses its own engine, and Jinja and other engines are available via plugins. Both look and work basically the same:
https://docs.djangoproject.com/en/4.0/topics/templates/
https://flask.palletsprojects.com/en...ial/templates/
The difference between Django and Flask is that Django offers more features, Flask a bit less, but they both provide very similar basic features. Django has a bit more defined project structure, and Flask can be more freestyle, but both of them have "standards" that people use and they also are fairly similar.
So, you can use Flask for lightweight stuff, Django for bigger projects. Depending on the situation, it may be faster to produce services with Django (thanks to ORM), but sometimes you may not need that much overhead and Flask would work just fine, and using Flask may be faster.
Imagine it like this. We use Django as a backend for our projects where we need a nice API (and want to use all of those fancy things like
https://www.django-rest-framework.org/ - PS: you don't need Django REST Framework to build REST APIs.. it just makes it more convenient) and then we use Flask for edge services/servers that allow us to control the OS, start/stop services, pull statistics, and so on.
Flask is not better than Django and Django is not better than Flask. As we both have acknowledged a few times is that "it all depends on the use case"... same applies to databases.
Imagine it like this.. Traditional SQL servers are general purpose servers that work for almost any use case. Nosql databases (MongoDB) are more specialized and have their own specific use.
06-01-2022 12:43 PM
#36
twinaxe (Senior Moderator)
Flask and Django both are frameworks themselves. With both, you write Python not some sub-language. Also "to code Django" doesn't make sense. Django does not have its own syntax or anything like this.
Of course Django isn´t an own language or has an own syntax.
It´s meant that Django itself is so big, has so many things already included and has a rather strict structure so that you have to learn very much about the framework itself and build your platforms in a specific way whereas with Flask you don´t have such strict structure.
06-01-2022 02:02 PM
#37
diplomat (Member)

Originally Posted by
twinaxe
Of course Django isn´t an own language or has an own syntax.
It´s meant that Django itself is so big, has so many things already included and has a rather strict structure so that you have to learn very much about the framework itself and build your platforms in a specific way whereas with Flask you don´t have such strict structure.
The only strict thing about Django is that "an app" must be in its own directory and it must be included in settings.py. What you see around the web is "the standard" structure that I was talking about. Flask has a similar "standard". As long as you have an app registered under "INSTALLED_APPS" then it's all good. It's a very minimal design pattern. You are welcome to be as creative with your files and folder as you like. Flask is "less strict" in that sense, yes.
I'm not forcing to you use Django if you don't want to. I'm just explaining that Django is not a big, horrible and scary thing. It's just a bit bigger when compared to Flask and that's why I'm saying Django is not better than Flask and Flask is not better than Django. They both have their uses and it's your job to decide which one you wish to use. I would strongly suggest you learn about both of them so you can make more accurate decisions in the future if you wish to continue with Python. They both are incredible!
But I would suggest you strongly think about the database and what kind of features you need now and in the future because changing databases is absolutely horrible.
06-01-2022 02:21 PM
#38
diplomat (Member)
https://hackr.io/blog/flask-vs-django
This article has a really nice comparison between Django and Flask that should explain things super well. And very conveniently it's the #1 search result for "django vs flask".
The only thing that I can't really agree on is: "For the same functionality, Django needs more than 2 times more lines of code than Flask.". That's not always true... due to the structure, there may be a few more lines yes, like authentication, CSRF, etc decorators, but that's not the case always. Since Django has built-in security system, it can actually reduce the number of lines of code you need to write vs Flask. But yeah, like most things, it depends on what you do.
06-01-2022 03:25 PM
#39
twinaxe (Senior Moderator)
It's just a bit bigger when compared to Flask and that's why I'm saying Django is not better than Flask and Flask is not better than Django. They both have their uses and it's your job to decide which one you wish to use. I would strongly suggest you learn about both of them so you can make more accurate decisions in the future if you wish to continue with Python. They both are incredible!
But I would suggest you strongly think about the database and what kind of features you need now and in the future because changing databases is absolutely horrible.
I also think that Django vs Flask is more or less a matter of personal liking as long as you don´t have a valid reason to use one or another.
They both can do incredible things and both frameworks are used by huge platforms as well.
With the databases however you really got me thinking because there I slowly understand that it´s often better to use another one than Mongo.
And this has direct impact on my previous decision against Django because if I decide to use MySQL or Postgres it would definitely make more sense for me to also use Django instead of Fast API for the ORM alone.
Will think about all these things
again but one advantage of chosing Django would be that @
larsometer also uses it so we could learn together a bit.
Thanks anyway for your input.
06-01-2022 03:28 PM
#40
larsometer (Senior Member)
Will think about all these things again but one advantage of chosing Django would be that @larsometer also uses it so we could learn together a bit.
Maaaaybe that will be the first time you can learn something from me and not only the other way around
06-01-2022 04:05 PM
#41
diplomat (Member)

Originally Posted by
twinaxe
I also think that Django vs Flask is more or less a matter of personal liking as long as you don´t have a valid reason to use one or another.
They both can do incredible things and both frameworks are used by huge platforms as well.
With the databases however you really got me thinking because there I slowly understand that it´s often better to use another one than Mongo.
And this has direct impact on my previous decision against Django because if I decide to use MySQL or Postgres it would definitely make more sense for me to also use Django instead of Fast API for the ORM alone.
Will think about all these things
again but one advantage of chosing Django would be that @
larsometer also uses it so we could learn together a bit.
Thanks anyway for your input.
If you wish to use Flask, please do that, it's super sweet. I would recommend trying Django because it does a lot for you (security, database management, something else important). So that will make your learning experience a bit faster and easier. I didn't want to intimidate you with all of this info. I just want to make sure you guys stay happy and continue learning and be successful. But to do that you need to do things right. Learn from my mistakes so you don't have to repeat them by yourself
Anyway guys, I'll be here for you and will do my very best to provide the best advice and help I can. @
larsometer and @
twinaxe you are always welcome to dm me or tag me or whatever.
06-02-2022 10:20 AM
#42
twinaxe (Senior Moderator)
Maaaaybe that will be the first time you can learn something from me and not only the other way around
We´ll see
But maybe it´s really a good idea to go that way.
We are both Python beginners, we want to work on similar projects, we are both Germans so that we can communicate easy so when we combine forces and "Hirnschmalz" we can probably learn pretty good from each other.
I would recommend trying Django because it does a lot for you (security, database management, something else important)
I totally agree about it, especially the security aspect is something I was thinking about anyway for quite some time.
Probably not that important yet but it can become very important later.
Database management is also a valid point to decide for Django.
Another big advantage is also that Django is maintained by one foundation.
With frameworks like Flask or Fast API you would have to rely on several/many different developers to maintain their libraries for functions that are already included in Django.
I just want to make sure you guys stay happy and continue learning and be successful. But to do that you need to do things right. Learn from my mistakes so you don't have to repeat them by yourself
I know, as you can see I am also not
that deadlocked in my decision and your arguments really got me thinking twice about some of my decisions.
I am thankful for every helpful criticism and in the end it´s still at me how I decide but especially the database stuff (and then in connection with Django) really makes sense so that I absolutey consider to switch my plans a bit
Anyway guys, I'll be here for you and will do my very best to provide the best advice and help I can. @larsometer and @twinaxe you are always welcome to dm me or tag me or whatever.
Thanks alot, sometimes the coding stuff can be quite confusing so that it´s good to have people to ask for advice.
Home >
>
Newbie Follow-Alongs