You can use SPSS without using any syntax. However, this often results in users having to do their entire data analysis all over again at some point. Just don't do it.
The good news is that routinely using Paste instead of Ok gets you a long way. Do keep in mind that the syntax pasted by SPSS is usually unnecessarily long and complex.
If you want to get a lot of work done with minimal time and effort, try to make a gradual transition from pasting to typing clean and simple syntax. The tutorials below will get you started.