Research Methods for Simple and Complex Systems
By David Alderoty © 2015
Chapter 4) Qualitative & Quantitative Research and Experimentation for Simple and Complex Systems
Over 3,900 words
To contact the author left click for a
website communication form, or use:
To go to the previous chapter left click on one of the following links:
For HTML) www.TechForText.com/R/Chapter-3
For PDF) www.TechForText.com/R/Chapter-3/PDF.pdf
If you want to go to the table of contents
of this chapter left click on these words
To Access Additional Information with Hyperlinks
After I complete a writing task, I select a number of websites from other authors, and link to them. The links are the blue underlined words, and they can be seen throughout this book. The in‑line links, such as the link on these words, are primarily to support the material I wrote, or to provide additional details. The links presented at the end of some of the paragraphs, subsections, and sections are primarily for websites with additional information, or alternative points of view, or in some cases to support the material I wrote. The websites contain articles, videos, and other useful material.
Brown bold text, like these words, represent quotes in this e‑book. You can access the original source, by clicking on a link presented just before a quote.
If a link fails, use the blue underlined words as a search phrase, with www.Google.com. If the failed link is for a video use www.google.com/videohp. The search will usually bring up the original website, or one or more good alternatives.
Simple Experimentation, and its Utility
The Simplest Concept of Experimentation
In the simplest sense of the terminology, experimentation means to test something to determine the results, or to try something to determine what will happen. People commonly do this throughout life, on a conscious or unconscious level, especially from infancy through early adulthood. When people do this, they usually have a general idea, or an assumption of the possible outcome of their trial, which is called a hypothesis. There experiment might confirm or refute their assumption.
Scientific experimentation, or formal experimentation, is usually more complex and organized then the above. It usually involves precise manipulation of entities that will be involved with an experiment. This can involve combining two or more entities, to determine how they will react. It can involve manipulating the environment in various ways to determine how it will affect the interaction between two entities.
The idea to keep in mind is informal experiments have value in everyday life, and even in planning formal experiments. This is explained below.
Before planning formal research projects based on experimentation, it may be useful to engage in quick informal experimentation first. This involves simplified, inexpensive, and less than perfect, experimental procedures, to make discoveries that can be evaluated at a later point in time with formal experimentation.
With simplified informal experimental procedures, it is usually possible to carry out a very large number of experiments with relatively little time, effort, and expense. This strategy can help you eliminate experimental research plans that are not likely to be fruitful.
The important idea to understand is informal experiments cannot be used to confirm a hypothesis, from the perspective of science or academic requirements. The purpose of quick informal experiments is to facilitate the development of research plans that involve formal experimentation.
Experimentation, and Simple Systems, With Related concepts
Experimentation for Very Simple Systems, such as The Systems of the Hard Sciences
For systems that are very simple, experimentation is usually a trial or test to obtain information, which may be either qualitative or quantitative. This can involve a number of experimental entities, but at the simplest level, it involves only two. An example is, combining two chemicals together, to determine the products that result from a chemical reaction. This example can involve the testing of a hypothesis, if the experimenter devised a conceptual framework to predict the results of the reaction. If the conceptual framework predicted the products of the reaction, the hypothesis is correct.
Simple Systems that are More Complex, and Experimentation
At a more complex level, experiments involving simple systems, can involve inventing a device, and testing it, to see if it functions as predicted. In some cases, a functioning device of this nature, may confirm a hypothesis.
Inventing devices and testing them are commonly done in the science of engineering, with the practical focus of creating new products. Usually, several experimental versions of the product, called prototypes are initially built, to test the functionality, manufacturing feasibility, durability, and practical utility of a product. When a product is complex, such as an automobile, the individual subsystems that will comprise the product, might be built and tested as individual components, before the prototypes are created.
Defining Qualitative & Quantitative Experimentation
What is Qualitative Experimentation
Based on the way I am using the terminology, qualitative experimentation, are experiments that are not based on mathematics. The experimental results are not evaluated with mathematics, but they are analyzed, and/or evaluated usually by human observations. However, in some cases, equipment may be used for evaluating results, such as electronic devices, chemical sensors, photographic devices, microscopes, etc.
With qualitative experiments, the experimenter might intuitively approximate quantities, ratios, and other mathematical concepts, without using math. This can involve a visual assessment, or the use of measuring equipment.
In actual practice, experiments in this category might involve a small amount of mathematics. For example, to calculate the quantity of ingredients needed for an experiment.
What is Quantitative Experimentation
Based on the way I am using the terminology, Quantitative experimentation, are experiments that involve mathematics, such as to measure quantities, and evaluate experimental results. Quantitative experimentation usually involves more than mathematical evaluations. These experiments usually include analysis and evaluation by human observations. Often, equipment may be used for evaluating results, such as photographic devices, microscopes, telescopes, electronic devices, chemical sensors, etc.
The Use of Mathematics, and Statistical Evaluations, In Scientific Investigations
Scientific Investigations, and Mathematics & Statistical Evaluation for Simple and Complex Systems
Mathematics is often used in scientific investigations to study both simple and complex systems. This includes experimentation, scientific observations, and surveys. Listed below, there are seven general examples where mathematics may be used in scientific investigations.
· To calculate Quantities or masses (This can involve calculating the quantity of ingredients, in terms of volume or mass, for an experiment. It can also involve calculations to determine the quantity of the products of a chemical or nuclear reaction. It can also involve measurements and calculations to determine the mass of black holes, stars, planets, living cells, molecules, atoms, subatomic particles, etc.)
· Calculations to compensate for inadequacies in measuring equipment (This can involve calculations to determine the average length, mass, or volume of an entity. For example, if you measure the height of an individual with a conventional tape measure a dozen times, you will probably obtain measurements that vary by a half-inch. Averaging all the measurements might provide the most accurate result. This strategy can be useful, when an entity varies in mass, volume, or length. It also may be useful when an entity is too small, or too large to obtain accurate measurements with currently available technological measuring equipment.)
· Calculating correlations (This very often applies to complex systems, such as human beings, to determine the relationship between two factors. For example, the relationship between social status and health can be determined by calculating correlation coefficients. However, with this example, it is necessary to devise a method of evaluating social status that can be converted into numbers. The same difficulty is encountered with health. This problem is not unusual with experiments and surveys that involve correlations.)
· Calculating the percentage of specific outcomes of an experiment (This often involves comparing the results in percentages of an experimental group and the control group.)
· Calculating the possibility that experimental results were caused by random chance (An example is improvement in a medical condition when an experimental drug was given to the patients. The results of well-designed experiments, with an adequate number of experimental subjects, usually have very low probabilities of occurring, because of random chance. Results of this nature are called statistically significant. Poorly designed experiments, especially if they involve a small number of subjects, may provide results that have a relatively high chance of occurring as a result of random chance. These results are called statistically insignificant. The above also applies to surveys.)
· Statistical assessments (This can involve evaluations to determine the significance of experimental results. This can also involve many other statistical evaluations, such as the breakdown of experimental results in terms of percentages, or presenting results in terms of an average. Statistical evaluations are used extensively with the complex systems of the social, psychological, and medical sciences, for experimentation, observations, and surveys.)
· Time, velocity, rate of change, energy input, energy output (This can involve calculations for biology, chemistry, physics, and engineering.)
Statistics and Complex Systems, and Related Concepts
Experiments with Complex Systems, and Statistics
Experimentation with complex systems usually requires statistical evaluation of the results. This is necessary because complex systems are often unique, one-of-a-kind entities. For example, human beings are complex systems, and there are no identical individuals, with exactly the same psychological and biological makeup. As a result, each individual may respond differently to the same experimental conditions.
Statistics and statistical evaluation of experimental results, is a complex subject, requiring special skills and training, when complex evaluations are required. It is not unusual for experimenters to higher mathematicians that specialized in statistics to carry out statistical assessments of their experimental results.
In the following subsection, there are some relatively simple concepts that involve statistics.
Averaging, and the Concealment or Loss of Data
Experimentation that involves simple and/or complex systems often involves calculating averages, as was explained in a previous subsection. In this regard, the important idea to understand is, when the average is calculated, the result generally represents less data, because some of the data is concealed or lost. The data that is concealed or lost when the average is calculated may be useless, meaningless, potentially useful, minimally useful, moderately important, or quite important or valuable. The following example will clarify this concept, using an extreme hypothetical case.
To illustrate the loss or concealment of data that can result from calculating the average, let us assume that there is a small village of 100 people, and the average yearly income is $200,000 a year. This suggests a wealthy community, which has no need for any type of public assistance. This would be true if all of the individuals had incomes that were above the poverty line. However, the average income of $200,000 a year does not prove that all individuals are wealthy. In fact, 99 of the 100 individuals may be below the poverty line. For example, let us assume that 99 individuals have an annual income of only $1000 a year, but one individual has a salary of $19,901,000. This averages out to $200,000 a year, but it conceals that 99% of the people are below the poverty line. See the following calculations:
When there are little variations in the individual numbers comprising an average, it will be unlikely that any important data will be lost. The greater the variations in the individual numbers comprising an average, the greater the chances of losing (or concealing) important data. When the variations are relatively large, to prevent loss of data, you can provide all of the numbers you used in calculating the average, or placing the numbers in groups. The hypothetical village, with 100 people, with an average salary of $200,000 a year, can be placed in two groups, as follows:
· Group-1 contains one person, average yearly salary of $19,901,000
· Group-2 contains 99 people, with an average yearly salary of $1000
To prevent the loss of useful data, it is a good idea to examine the individual cases that are involved in an experiment. This involves looking for individual cases that display large variations from the average, and then attempting to place them into relevant groups.
Another strategy involving three steps, to evaluate the possibility of data loss when taking the average is presented below, using the example of the village with 100 people, with an average salary of $200,000 a year.
· Step-1) Divide the numbers comprising the average in half, so that there are two subgroups with equal quantities. This should be done randomly, without looking at the numbers. With the example of the hypothetical village of 100 people, this results in subgroup‑1 with 50 people, and subgroup‑2 with 50 people.
· Step-2)
Calculate the average of subgroup‑1 and subgroup‑2.
With the hypothetical village, the average of subgroup‑1
is 
and
subgroup‑2 is 
· Step-3) Examine the average of subgroup‑1, and subgroup‑2. If the average of each subgroup is approximately the same, there is no significant loss or concealment of data, with the average calculation for the entire group. However, if there is a modest to large difference in the average calculation for subgroup‑1, and subgroup‑2, there is loss or concealment of data, when the average is calculated for the entire group. With the hypothetical village, the average of subgroup‑1 is $1000 and the average for subgroup‑2 is 399000. This indicates that there is loss or concealment of data, when the average for the entire group, of $200,000 is presented.
For greater precision, the technique described above, can be repeated with subgroup-1 and subgroup-2. This will result in four additional subgroups, which I am designating as subgroup-A1, and subgroup-A2, subgroup-A3, and subgroup-A4. Then the average can be calculated for each of the four subgroups.
The idea to keep in mind is to examine any calculation that is based on the average, to determine if there is any lost or concealed data. This relates to how much the numbers used to calculate the average varies. If there is little variation between the numbers, there will be no loss of concealment of significant data. If there is a great deal of variation between the numbers, the average will represent loss or concealment of data.
The strategies presented above are simple ways of dealing with the potential loss or concealment of data that can result from calculating the average. On the following websites from other authors, there are statistical techniques that are more complex, which can be used to deal with this challenge. 1) Statistical Variance, 2) Standard Deviation and Variance. The following are YouTube videos, and related webpages. On each of these webpages, there are a number of good videos on statistics. 3) Video: What is Variance in Statistics?, 4) Video: Statistics - Calculating Variance, 5) Video: Standard Deviation and Z-scores, 6) Video: What is a "Standard Deviation?"
More Complex Experimentation, with Controls
Experimentation with certain types of systems requires two groups of experimental systems, such as two groups comprised of animals, or people. This is especially the case with the complex systems of the social, psychological, and medical sciences. One group is exposed to the experimental conditions, and the other group is not exposed. The group that is exposed to the experimental conditions is usually called the experimental group, and the group that is not, is usually called the control group.
A control group or controls are usually essential with complex systems. However, they may be unnecessary when experimentation is based on simple systems, such as chemicals, mechanical components, electronic circuits, etc. In general, controls are often unnecessary for the simple systems of the hard sciences, but this is certainly not always the case.
What I mean by controls, involves the process of comparison of the experimental group, with the control group. At the most basic level, this involves systems (such as people) that are exposed to experimental conditions, and systems that are not exposed. This will be clarified with the following examples, which are based on medical science. (Keep in mind that the concepts that I am presenting, CAN APPLY TO ANY EXPERIMENT, especially if it involves complex systems, such as animals or people.)
When a new medication is tested, there will be two groups of individuals. One group will receive the medication, which is the experimental group. The other group will be given a placebo, which is an inactive substance, such as a capsule filled with sugar or starch. This group is usually called the control group, or the placebo group. The individuals involved in the experiment do not know if they are in the experimental or placebo group. In some cases, the individuals carrying out the experiment do not know which group is receiving the medication and which group is receiving the placebo.
The side effects and effectiveness of the medication is determined by comparing the experimental group with the placebo group. This is necessary because circumstances surrounding the experiment can influence results, such as psychological factors that relate to the individuals involved with the experiment. This influence can produce adverse side effects, and/or improvements in a medical condition, which are not related to the medication that is being tested. This can distort the results of the experiment. The following four examples will illustrate how a control group with a placebo, can be used to eliminate this distortion from the experimental results.
· Example 1) Let us assume 3.1% of the placebo group vomit, and 3.1% of a group that is receiving a new experimental medication vomit. With this example, the vomiting is probably not from the experimental medication. This is because it occurred in the placebo group at the same level as in the experimental group.
· Example 2) Now let us assume 1.2% of the placebo group vomit, and 4.3% of a group that is receiving the medication vomit. With this example, the vomiting is probably caused by the experimental medication. This is because it occurred at a higher rate in the experimental group, and the lower rate in the placebo group.
· Example 3) Let us assume 65.2% of the placebo group improve and 65.2% of a group that is receiving an experimental medication improve. With this example, the improvements are most likely not from the medication, because the same rate of improvement was obtained in the placebo group.
· Example 4) Now, let us assume 5.2% of the placebo group improve and 68.4% of a group that is receiving an experimental medication improve. With this example, the improvements are most likely from the experimental medication. This is because the improvements in the experimental group was much higher than the improvements in the placebo group.
Positive and Negative Placebo Effect
A Note for the Examples Presented above, And the Placebo Effect
The improvements and the adverse side effects that were not related to the experimental medication in the above examples, is often called the placebo effect for improvements, and the nocebo effect, for adverse side effects. In this e‑book, I am using the term negative placebo effect, for the words nocebo effect.
The placebo and nocebo effects are presumed to be psychological by most sources. However, in this e‑book I am using the terms to represent all of the following: psychological effects, physiological effects initiated by psychological dynamics. In addition, I am including physiological effects that are inadvertently initiated by factors in the environment, which are not related to the experiment. This can involve factors that affect the individuals participating in the experiment, which maybe unknown to the experimenter. This might involve lack of adequate food, lack of sleep, excessively high or low room temperatures, lack of ventilation, and many other factors.
When the placebo effect is positive, it indicates an improvement, which is usually the result of psychological dynamics, such as positive thinking. Improvements caused by the placebo effect usually are temporary. These improvements may involve simply ignoring symptoms of a medical condition, or the masking of symptoms as a result of a positive attitude, which may last a few hours or even a few days.
When the placebo effect is negative (nocebo effect) it may appear that a medical condition is worsening, or adverse side effects, might develop. This can be the result of psychological dynamics, such as anxiety, or fear, associated with the possible risks of being a subject in an experiment. The adverse effects are usually temporary, and disappear after the experiment has been completed.
Most medical conditions either spontaneously improve, or get worse. These changes can also be associated with the positive or negative placebo effect. (This situation is not limited to experimental research. It happens in everyday life, when people attribute an improvement in a medical condition to medical treatment, drugs, or over-the-counter remedies or supplements. The improvement may be the result of the body’s natural healing mechanisms, and not related to the efforts to treat the medical problem. This is probably less likely to happen with serious medical conditions, and more likely to happen with minor medical problems, such as the common cold.)
In general, the positive and negative placebo effect can interfere with experimentation, when human beings are involved as subjects. This can happen with medical and psychological experiments. However, with proper experimental design, involving a control group and an experimental group, the interference from the positive or negative placebo effect can be filtered out of the results of an experiment.
*Is there a Placebo Effect with Animals?
Experimentation with animals also requires controls, because animals are complex systems that can be unpredictable. This raises the question, is there a negative or positive placebo effect with animals? Animals cannot understand that they are in an experiment. They have no way of knowing that they may be receiving medication that may help them, or harm them. As a result, animals will not display the same beneficial or adverse effects that human subjects might display, as a result of the psychological impact of the above. However, animals are affected by different set of psychological dynamics, which can result in a positive or negative placebo effect.
Animals involved with an experiment are likely to sense that something is going on in their environment that they do not understand. This is especially the case if they are placed in strange cages, with electronic devices, and physically restrained to receive hypodermic injections. This is likely to result in anxiety and fear. Thus, there may be placebo effects, when animals are used as experimental subjects. This will be clarified with the following examples:
· The cages used for the experimental animals (Cages can either produce a positive or negative placebo effect. This problem is solved by putting the control group in the same type of cages as the experimental group.)
· The food given to the experimental animals (This can result in either a positive or a negative placebo effect. This problem is solved by feeding the same food to the control group.)
· The pain associated with hypodermic needles (This occurs when an investigational drug is injected into the experimental animals. This might result in a negative placebo effect. The problem is solved by injecting the control group with a physiologically inert chemical, such as a saline solution.)
· The overall atmosphere of the laboratory (This can have a positive or negative placebo effect. This can involve the presence or absence of noise, odors, ventilation, temperature variations, and unknown factors, in the laboratory. This problem is solved by putting the cages of the control group right next to the experimental group.)
· The attention given to the animals (This can result in a positive or negative placebo effect. This difficulty is solved by giving the control group the same type and level of attention as the placebo group.)
· The screaming from the animals, caused by restraining them for injections, examinations, or sacrifice (This might have a negative placebo effect. This is especially the case, with higher animals such as apes or monkeys. This problem can be minimized by placing the controls and the experimental group in the same room. Ideally, this room should be isolated from the main laboratory where examinations, injections, and sacrifices are carried out.)
1) The Qualitative Experiment in HCI: Definition, Occurrences, Value and Use, 2) Drawbacks of Qualitative Evaluation in Chemistry Experiments, 3) Quantitative, qualitative & experimental research, 4) Qualitative Inquiry in the History of Psychology, 5) A Methodology for Discovery in Psychology and the Social Sciences. Gerhard Kleining & Harald Witt, 6) The history of the Quantitative experimentation and qualitative natural observation, 7) Natural and Field Experiments: The Role of Qualitative Methods, by Thad Dunning Department, 8) METHODOLOGICAL EXPERIMENTATION AND QUALITATIVE RESEARCH DESIGNS Mirka Koro-Ljungberg, 9) A Qualitative Experiment To Analyze Microbial Activity in Topsoil Using Paper and a Handmade Reflection Photometer, 10) Quantitative cell biology: the essential role of theory, 11) The history of the Quantitative experimentation and qualitative natural observation
To go to the first page of this
chapter left click on these words
Below is the hyperlink table of contents of this chapter. If you left click on a section, or subsection, it will appear on your computer screen. Note the chapter heading, the yellow highlighted sections, and the blue subheadings are all active links.
Chapter 4) Qualitative & Quantitative Research and Experimentation for Simple and Complex Systems
To Access Additional Information with Hyperlinks
Simple Experimentation, and its Utility
The Simplest Concept of Experimentation
Experimentation, and Simple Systems, With Related concepts.
Experimentation for Very Simple Systems, such as The Systems of the Hard Sciences
Simple Systems that are More Complex, and Experimentation
Defining Qualitative & Quantitative Experimentation
What is Qualitative Experimentation
What is Quantitative Experimentation
The Use of Mathematics, and Statistical Evaluations, In Scientific Investigations
Scientific Investigations, and Mathematics & Statistical Evaluation for Simple and Complex Systems
Statistics and Complex Systems, and Related Concepts
Experiments with Complex Systems, and Statistics
Averaging, and the Concealment or Loss of Data
More Complex Experimentation, with Controls
Positive and Negative Placebo Effect
A Note for the Examples Presented above, And the Placebo Effect
*Is there a Placebo Effect with Animals?
To go to the first page of this
chapter left click on these words
If you want to go to the next chapter
left click on one of the following links
For HTML) www.TechForText.com/R/Chapter-5
For PDF) www.TechForText.com/R/Chapter-5/PDF.pdf
If you want to see a list of all
the chapters in this e-book go to