Let's make an analogy with Quantum and Classical Mechanics. It is beyond doubt that the Classical description of physical reality is fully deterministic, and that we indeed observe that determinism in everyday experiences like that of gravity. On the other hand, if we delve into the Quantum realm, we find ourselves before an essentially indeterministic reality, which again is experimentally assessed and confirmed. So, it appears that, somehow, when we bring many "free" quantum objects together to form macroscopic classical objects, the bulk observed result of the multiple degrees of freedom actually becomes constrained in a deterministic way.
Now, one can consider each individual as a free agent with actual free will, and still observe deterministic emergent patterns when one takes into account the bulk effect of many individuals making free choices, or even when looking at the cumulative effect of many individual choices.
It's true that I can only act based on what I already know. However, I know enough to give myself more than one choice in many scenarios. Some argue that my choice even then is inevitable. However, we have no conclusive proof of this either way. I think it's more likely that free will does exist, but within parameters.
If you define "free will" in the libertarian sense, then it is incompatible with determinism because according to libertarianism, acts of the will are not determined by any antecedent condition, but according to determinism, everything is determined by antecedent conditions. However, "free will" in the compatibilist sense is not inconsistent with determinism because according to compatibilism, our actions are determined by our strongest desire or inclination.
It's all about perception and consciousness. Everything has a 100% chance of happening from an omniscient point of view, but not to us. From a subjective standpoint, we do have free-will because it's most logical that we say we do. Yes it's somewhat of a semantics debate but actions are subjectively evitable rather than inevitable. See Dennett. The future is set, but evitability still exists.
And we will never be able to. It doesn't matter if our lives have a set course, we are unable to perceive it as such and the choices we make, to us, are ours to make. You couldn't argue that we aren't somewhat controlled in how we live, we go through life reacting to things happening around us, but how we choose to react is up to us. There is no invisible force that we can sense that restricts us in what we potentially are capable of deciding.
Free will and determinism can coexist depending on the situation and mindset of the individual. Believing that outcomes are already set can allow the individual to in turn believe that he has the ability to produce that come or change it. The future is unknown, and the we believe outcomes are certain, they are the result of our actions.
Determinism, or the idea that prior states create circumstances that mold and narrow our available options does not mean that we could not, at any give point, act in ways that imply free will. For example, poverty is generational and given a state where one is raised in poverty one is most likely to end up in poverty, however one may choose to navigate their circumstances in many different ways, such as remaining in poverty, refusing or accepting government assistance, attempting to move up the social/cultural hierarchy, etc. Now, it could be argued that determinism limits and determines even those choices, but that does not mean that free will is not a factor or option.
Free will is defined as the ability to have control over one's will and consequently one's actions. If reality is ultimately deterministic, there is no room for free will, since if each past state of the universe determines the next, it is implied that one can predict everything that follows.
They are polar opposites. Free will means that you act at your own discretion, where as determinism means that you believe everything is predetermined. If everything is predetermined, than no matter how hard you try to exercise free will, in the end your decision to do so what already predetermined. It is IMPOSSIBLE to act freely if your actions are not by definition, free.
I think that to some extent determinism is akin to destiny. If you believe in destiny, it is hard to believe in free will. Because then it seems like there is a larger force in control of what we do. I think free will is completely independent from that situation.