So I'm just wondering how hard is it really
There are some things to consider and to be careful about but that goes with almost anything you do in life. But as others have already said, it's mostly like lego.
This is roughly how it goes:
1. order your hardware: case, mainboard, CPU, GPU, RAM, SSD/HDD, power supply. Those are the basic components with each coming with everything you need to install it. Your case comes with any screws and nuts and slots you're going to need to tighten/install your HDD/SDD. The power supply comes with all the cables you need to connect your hardware to. Your mainboard comes with all the cables you need to attach to your HDD/SDD. The CPU comes with a default fan and cooler and thermal paste.
It is sometimes handy to a few extra small components like extra thermal paste, extra cables, and so on.
2. You put everything onto your mainboard like lego, everything should go with minimum effort and almost zero force. The tightener for the CPU socket and the RAM naturally requires just a tiny amount of force since it tightens things. There's also only one slot for each respective component. You can't install you GPU in the wrong socket. You can't install the RAM the wrong direction. If in doubt: check the manual.
(You can also install your mainboard first into your case and then put everything on it afterwards. It depends on preference and how much space you have inside your case to fiddle around.)
3. Some questions may arise, like: I have four RAM banks and only two bars, do I just put them in how I want to? This is answered in your mainboard's manual. Usually mainboards have banks like A1, A2, B1, B2 and you simply match the number in increasing letters (e.g. 1 bar to A1, the other one to B1). But again, this is well described in your mainboards manual. Another question might be: how do I install my case fans and the LED power light and the front USB panel and such? This is also well-documented in your mainboard's manual. Many mainboards if not all have also some imprints at certain slots and sockets, saying something like "CPU_FAN" or "FAN_1", "FAN_2" or "HDD LED" or "AUDIO PANEL".
(Now, many cases allow the cables to run through a dedicated compartment behind the mainboard so your inner case looks tidy and neat and allows the best possible airflow with no cables obstructing it. Check your case before you install anything to see what options you have, also refer to the manual if in doubt).
4. Install/click the I/O shield into the back of your case with the big open slot.
Put everything into your case and tighten the mainboard with the designated screws onto the designated base.
5. Connect the cables.
Power supply to: Mainboard, GPU, HDD/SDD (and DVD/BluRay drive)
SATA cables from mainboard to HDD/SDD (and DVD/BluRay drive).
CPU and case fans to mainboard.
This is an example from my mainboard's manual on how to connect the case's cables to the mainboard; it doesn't get more complicated than this, it's already the maximum intricacy:
6. Switch on your power supply, hit the boot button.
So the things you need is common sense, the manual of any given component, a bit of patience and/or one or two YouTube videos.
This whole process, if it's the first time you do it, should take you no longer than one or two hours.
If problems arise, don't panic, don't try something funny you don't know anything about, but check the internet, ask people who know it and/or watch a YouTube video on this.