I believe deer will remember a place of danger... Say you were walking thru a woods and saw a big hole under a tree and as you passed by a bear ran out of it at you... Every time you pass that spot again, you would circle way around "just in case" I would imagine you would not forget for a long time... Young bucks and does grow to be old bucks and does. I fully believe they are capable of remembering where they encountered danger.
The smell of a human is alarming. This by itself would probably only create concern for a short while... But repetitive human scent is something to worry about... If you saw a bear track in my yard, you would look over your shoulder and feel nervous for a moment or two. You would forget about it after a while... If you see new tracks every time out, I think you would be worried a little about the bear... Maybe bears are a bad example because they don't really hunt people. Imagine a tiger maybe.
For experience, I can relate this to my public land hunting. I know where several good spots are that someone else sits and sits often. Generally the deer im after skirt around those stands 40 or more yards down wind.
It would be a lot easier to right past the stand, but even in the summer I find the trails still circling around the "known" stand site.