First.) Kata has been the traditional method of military knowledge transmission in both the East and the West from the time of organized military battle. The Manual of Arms (for swords, pikes, spears, shields, firearms, etc.), Parade Ground Drills/Phalanx movements, Dry Firing, House and Room Entry and Clearing, Close Combat Drills, and Hand to Hand drills. ALL of these are Kata in reality.
Second.) The Orient skipped the birthing pains of the Industrial and Political revolutions, that the West endured; and went straight from Feudalism to modern political thought and industrialization - skipping from the 15th to the 20th centuries. As a result, their cultural past was present when Nationalism and pride in ones' native culture took hold globally. The East maintained the tradition of kata training; whereas in the West the tradition had turned to technique. In the East techniques CAME from Kata. It was an extension of the process involved in kata training.
Next.) Kata training involves four stages.
A.) Simple. Trying to do the movements correctly; and in the right order. This is the entry level and requires much and extensive hard work to succeed. Most fail at this level. They do not have the patience and persistence to correctly learn the movements. They are too mentally challenged to complete this stage of understanding.
B.) Hints/Clues. Every movement in a kata is a Hint or a Clue as to its TRUE meaning. True meaning? Yes! Kata was meant to be misunderstood by those observing it. That is why even the names of the movements are meant to misdirect the observer. Block, side punch, etc. These, in their simplest meaning, match what 'appears' to be happening in the kata; but they are roadblocks to the learning process. Similarly, kata (in the beginning) repeat to the right and left sides, creating the impression that we need to practice BOTH left and right sides equally. This causes the uninformed to waste valuable time practicing that which will cause him to hesitate in a real self defense situation. In order to understand what their applications could possibly be, you must think about the movement: in reverse, as it applies to the sides, to the back, front, hands open or closed. Understanding all of these movements lead to success in the final stage of kata training.
C.) Oral Tradition. Most Okinawan, Japanese, Korean, and Chinese systems did NOT exist before WW2. Without the benefit of a lineage that stretches back before the 1900's you are, in essence, re-creating the wheel. You do not have the knowledge of what really works in a fight and what is blarney. A TRUE Instructor, gives you ideas that you never thought of, reminds you of what you have forgotten, answers your questions with common sense, practical, things that you can do; and, are designed to protect YOU from harm - physical, legal, mental, or moral.
D.) Secret. Kata has prepared you for the physical and mental processes so you can progress in discovering what the kata truly can mean. A secret is merely something you do not know. From the movements the kata imbues in you, you can work out self defense techniques against the common violence you will encounter in daily life. Kicking and punching in a dueling sort of way is fighting. If you cannot fight, you cannot do self defense. But, animals fight. We are supposed to be gentlemen. High class people not low class thugs. Self defense is much harder to pull off and requires greater skill sets than mere punching or kicking.
And, believe it or not, our self defense techniques are taught, you could very well say, as "mini-kata". You do them in the air, without benefit of a partner while learning them, during testing, and when you have no workout partner. After you first learn them, it is CRITICAL that you work with a partner. Once you understand how they work (and, that, can ONLY be achieved while working with someone), then, and only then, will you be be able to correctly work in the air with your 'mini-kata'.