Re: Online scientific discourse is broken and it can be fixe
For the record — just so there is no confusion — I am not advocating for any claim within that article other than the existence of a pattern of professionalization within the sciences. All that I am doing here, at this current point, is very simple: I am trying to shine light on systemic interdependencies.
What I would most like to achieve at this point is for people to start viewing the problematic patterns we observe in scientific theory and discourse within a larger educational framework which encompasses not just the technical arguments, but also the systems we use to teach and discuss science. So, the first objective is to simply show that these cross-domain interdependencies between educational technique/"infrastructure" and scientific belief actually exist, and that the values play an important role in mediating this interconnection. The values are unseen forces — a "hidden curriculum" — which are being taught in parallel to the content of science itself. This hidden curriculum is rarely discussed amongst advocates of conventional science even though it would somewhat plainly appear to exhibit great influence upon scientific consensus. If it can more-or-less be established to exist, then that can open the door to conversation(s) which I feel are necessary in order to meaningfully discuss science education reform.
Plasmatic
Re: Online scientific discourse is broken and it can be fixe
So, you reject the notions in that article that equate induction with intuition, and dont intend to connect this conception of induction to "creative meaning making" then? That is, you accept that induction is an objective relation between subject and object as much as deduction is? (And that emotion has nothing to do with either because its not a tool of cognition?)
pln2bz
Re: Online scientific discourse is broken and it can be fixe
This Remains an Unfolding Situation
One of the things I wish to emphasize — based upon personal observations via interactions with others online — is that the case that Jeff Schmidt makes in Disciplined Minds remains largely unknown amongst the public, amongst advocates for conventional theories, and importantly, to graduate students. I would propose that not only does this lead to unique ramifications for each of these three groups, but a broader awareness of Jeff's basic thesis could potentially open the door to a larger consensus on how to proceed with education reform within the sciences.
Take, for example, a September 2013 resignation letter penned by a graduate student titled, "An Aspiring Scientist's Frustration with Modern-Day Academia: A Resignation". The student appears to invoke many of the problems pointed out by Schmidt 13 years earlier in 2000 — but apparently without any reference to Schmidt's work …
Dear EPFL, I am writing to state that, after four years of hard but enjoyable PhD work at this school, I am planning to quit my thesis in January, just a few months shy of completion …
… the essential motivation stems from my personal conclusion that I've lost faith in today's academia as being something that brings a positive benefit to the world/societies we live in. Rather, I'm starting to think of it as a big money vacuum that takes in grants and spits out nebulous results, fueled by people whose main concerns are not to advance knowledge and to effect positive change, though they may talk of such things, but to build their CVs and to propel/maintain their careers …
The good, healthy mentality would naturally be to work on research that we believe is important. Unfortunately, most such research is challenging and difficult to publish, and the current publish-or-perish system makes it difficult to put bread on the table while working on problems that require at least ten years of labor before you can report even the most preliminary results. Worse yet, the results may not be understood, which, in some cases, is tantamount to them being rejected by the academic community …
(Notice that we have yet another instance here of the system of communication itself influencing our beliefs within theory, as well as a pattern which is very familiar to Electric Universe proponents: rejection of an idea due to a failure to comprehend it. We might also wonder if there exists a connection between these two problems ...)
I cannot help but get the impression that the majority of us are avoiding the real issues and pursuing minor, easy problems that we know can be solved and published. The result is a gigantic literature full of marginal/repetitive contributions …
Unfortunately, not only does this lead to quantity over quality, but many researchers, having grown dependent on the bandwagon, then need to find ways to keep it alive even when the field begins to stagnate. The results are usually disastrous. Either the researchers begin to think up of creative but completely absurd extensions of their methods to applications for which they are not appropriate, or they attempt to suppress other researchers who propose more original alternatives (usually, they do both). This, in turn, discourages new researchers from pursuing original alternatives and encourages them to join the bandwagon, which, though founded on a good idea, has now stagnated and is maintained by nothing but the pure will of the community that has become dependent on it. It becomes a giant, money-wasting mess.
And although I will have much more to say about the observable problems in science education and discourse, this last point brings up a potential solution that is worth exploring in more depth. It also gives us a chance to talk about what constitutes "state of the art" in visualizing the structure of science today.
They Make for Very Pretty Pictures, But Did Anybody Think to Check that They Work?
Our brave-but-anonymous graduate student is proposing the existence of an unfortunate chain of events here which they claim starts with this publish-or-perish system, more formally known as scientometrics. Scientometrics is a graphable system for tracking reputation in science, by using the metric of citations (I will be discussing the general notion of graphs later in my focus on a possible solution). In this view, the value of a paper is simplistically equated with how many times it has been cited by other papers within other prestigious journals. I'm aware of at least three observable problems with scientometrics as a system for diagnosing or visualizing value in science:
Researchers sometimes refer to previous research to indicate the source of certain influences or to identify past work that they want to develop further. But they may also cite previous work in order to argue against it or maybe even refute it entirely. And sometimes sources are referred to out of tradition or [routine — like] because everybody else in a field seems to do it.
'The conclusion of this is that the number of times research is cited is a rather poor indicator of its scientific quality nor that more citations automatically means high quality, says Nelhans.
As a result of the so-called citation culture that has emerged in the scientific community, an increasing number of researchers have started to present their studies not only with the obvious goal of promoting the content, but also with an aim to attract as many citations as possible. The purpose of this is to gain acknowledgement in the scientific community and secure research funding.
On the other hand Nelhans argues: You can on the other hand say that a cited article has been 'used' by the later literature and that it therefore is 'visible'. But this has to be done carefully since citations only show up for certain publications, namely articles published in certain peer-reviewed and internationally distributed scientific journals.
Nelhans' thesis points to how the awareness of the effects of citations on research has led to them being perceived as hard currency in the scientific community, from the national level down to the individual researcher at his or her department.
'The problem is that citation statistics offer a complex measurement that hides at least as much information as it reveals. It is therefore important to see the whole extent of this phenomenon and not treat citations as an automatic measure,' says Nelhans, who urges decision-makers to be more careful when basing allocation of research funding on citation statistics.
And this brings us right to the second thing you need to know about scientometrics ...
(2) We already have good reason to believe that rewards can narrow focus, and whether or not that is a desirable approach depends upon the type of problem we are trying to solve.
This unexpected experimental result has been widely disseminated in various books on creativity. I will repeat the version we can find in Daniel Pink's book, Drive:
Creativity
For a quick test of problem-solving prowess, few exercises are more useful than the candle problem. Devised by psychologist Karl Duncker in the 1930s, the candle problem is used in a wide variety of experiments in behavioral science. Follow along and see how you do.
You sit at a table next to a wooden wall and the experimenter gives you the materials shown below: a candle, some tacks, and a book of matches.
The candle problem presented.
Your job is to attach the candle to the wall so that the wax doesn't drip on the table. Think for a moment about how you'd solve the problem. Many people begin by trying to tack the candle to the wall. But that doesn't work. Some light a match, melt the side of the candle, and try to adhere it to the wall. That doesn't work either. But after five or ten minutes, most people stumble onto the solution, which you can see below.
The candle problem solved.
The key is to overcome what's called functional fixedness. You look at the box and see only one function as a container for the tacks. But by thinking afresh, you eventually see that the box can have another function as a platform for the candle. To reprise language from the previous chapter, the solution isn't algorithmic (following a set path) but heuristic (breaking from the path to discover a novel strategy).
What happens when you give people a conceptual challenge like this and offer them rewards for speedy solutions? Sam Glucksberg, a psychologist now at Princeton University, tested this a few decades ago by timing how quickly two groups of participants solved the candle problem. He told the first group that he was timing their work merely to establish norms for how long it typically took someone to complete this sort of puzzle. To the second group he offered incentives. If a participant's time was among the fastest 25 percent of all the people being tested, that participant would receive $5. If the participant's time was the fastest of all, the reward would be $20. Adjusted for inflation, those are decent sums of money for a few minutes of effort a nice motivator.
How much faster did the incentivized group come up with a solution? On average, it took them nearly three and a half minutes longer. Yes, three and a half minutes longer. (Whenever I've relayed these results to a group of businesspeople, the reaction is almost always a loud, pained, involuntary gasp.) In direct contravention to the core tenets of Motivation 2.0, an incentive designed to clarify thinking and sharpen creativity ended up clouding thinking and dulling creativity. Why? Rewards, by their very nature, narrow our focus. That's helpful when there's a clear path to a solution. They help us stare ahead and race faster. But if-then motivators are terrible for challenges like the candle problem. As this experiment shows, the rewards narrowed people's focus and blinkered the wide view that might have allowed them to see new uses for old objects.
(3) We also know that the process of peer review can act as a filter for worldviews which diverge from the conventional. Much has been written on this by others, and the implication is that scientometrics appears to be fundamentally based upon an incomplete set of ideas which is incapable of comprehensively representing promising new ideas in science. Judging the adherence of these ideas to the values of science can help act as a guide for thought for ideas which are new to the professional scientific community, but what do we do about apparent conflicts in values which become more visible when we look at different communities of thinkers?
The System-of-System View of Scientific Discourse
This is where I think that things start to get interesting ...
What I would like to suggest is that we might consider the utility of compartmentalizing the discourse such that it more naturally emulates the types of communication patterns we already observe in the real world. In other words, the flows of information which we would expect to constitute effective scientific communications should arguably be modeled by our scientific social network. If such a design framework was adopted, then it would offer an opportunity to accommodate both activities — both reputation-based (convergence-dominated) inquiry as well as the elaboration of creative, divergent models, possibly based upon competing worldviews — without either activity encroaching upon or dominating the other. And for those who have read books on creativity or innovation — a very popular topic in the business world — they may recognize the absolute necessity of both processes to effective innovation.
Once we get into the details of visualization, I'll attempt to construct a speculative simplest solution which attempts to aligns these values domains with the levels of scientific discourse itself (namely: concepts, propositions, models and worldviews). But, for now, I just wanted to plant the much more general idea that if we permit ourselves to think like information architects designing a scientific social network in response to critiques involving values, then the competing values inherent to science education can be encapsulated and even directed using interacting-yet-independent domains within a larger system of systems. In such a system, there is the potential to restructure what might normally be clashing cultures which refuse to generally talk with one another into a system of weakly interacting cultures that can possibly benefit from one another's existence.
After all, although it is not currently the conventional view, one of the lessons we've learned from the Electric Universe is that …
"Heretics are absolutely necessary in science. I think it's no exaggeration to say that virtually all of the major breakthroughs that have occurred in the history of science have occurred through heretics, and have started out as heresies. Most of the progress in science requires a willingness to be wrong. Somebody has to risk being ostracized by the scientific community to advance science." (Nobel Prize Winner, Medicine - Linus Pauling - Heretics of Science - YouTube, 11:46 - 12:19)
So, it stands to reason that we can potentially use this idea of weakly interacting values domains to more properly facilitate interactions between communities of thinkers — and in the process better facilitate changes to scientific belief on a timeline which might better accommodate our own short existences on this planet.
Plasmatic
Re: Online scientific discourse is broken and it can be fixe
Chris, has it occured to you that the proposed solution to the candle problem is not in fact an instance of what the declared goal was? In the proposed solution the tack box is attached to the wall and not the candle. Its instructive for future reference that this context dropping incentive to push an anti profit mindset, turns on an evasion of meaning. That is, one is supposed to accept that attaching a box to a wall is conceptually equivalent in meaning to attaching a candle to a wall. This will become important the more explicit ones view of concepts/knowledge is laid out.
The goal is to convince one that the solution to "attach the candle to the wall" is to treat BOTH box AND candle as interchangeble. In other words, to be creative, abandon logic....and meaning.
Edit: in other words, to Pink, what "narrows our focus" is attaching meaning to words.
Edit: if the test said "elevate the lit candle above the table without it dripping on the table" I bet the time to task would not serve Pink's purpose.
pln2bz
Re: Online scientific discourse is broken and it can be fixe
The goal is to convince one that the solution to "attach the candle to the wall" is to treat BOTH box AND candle as interchangeble. In other words, to be creative, abandon logic....and meaning.
I can definitely see why you might be inclined towards skepticism. It's a counter-intuitive finding, and to Pink's credit, he distances himself from the extreme interpretations. My own personal observations of the Bad Astronomy and Universe Today forum seemed to plainly suggest that creative problem-solving is a delicate process — a bit like a candle flame, actually — which will predictably extinguish if it is not carefully protected. In terms of values, I suspect that we might expect to see only a slight overlap between the set of values associated with thinking like a scientist and thinking like an artist — but we should not interpret that as reason to believe that creativity is something we can simply remove from scientific thinking without long-term consequences.
My own personal observation is that there currently exists a tendency for university-educated astrophysicists to exhibit hostility towards manifestations of creativity within the scientific domain (and my tendency is to induce that this is a pattern associated with our science graduate programs, more generally). This either/or mentality likely leads to problems. I am advocating that we try to find way(s) of thinking which treat creativity and science with a both/and approach. And one way to do that might be to think of creativity as an essential influence upon the theory-making process in the long term, which if left out as a value in scientific discourse and education, ultimately leads to consequences like the ones which our anonymous graduate student identifies — such as the bandwagon effect.
The Borders Between Concepts
There is also reason to believe that something very fundamental is at play in the candle experiment that pertains to the dynamic nature of conceptual borders in light of a changing context (It seems that you might have actually sensed this with your edit …).
We need not start from scratch on this point. This is actually the topic of Peter Gardenfors' third chapter in his book Conceptual Spaces. He starts that chapter off with the following quote ...
This capacity to predicate is absolutely central to concept-using creatures. It means that the creature is able to identify the common property which two or more objects share and to entertain the possibility that other objects also possess that property. That is, to have a concept is, among other things, to have a capacity to find an invariance across a range of contexts, and to reify that invariance so that it can be combined with other appropriate invariances. (Kirsch, D. 1991. "Today the earwig, tomorrow man?" Artificial Intelligence 47: 161-184)
And he next notes that in most semantic theories or theories of concept formation, no distinction is made between properties and concepts. He then proposes a theory which invokes geometry in the brain's encoding of conceptual representations (hence the title of his book …).
I could not possibly do his theory justice here, given how careful he attempts to be with his justification, but what I can offer as a way out of the predicament for now — since we are still describing the apparent problems of scientific education and discourse — is to take a quick look at some experimental data which suggests that the properties of concepts exist in relation to an apparent prototype, and that the borders between concepts may change in relation to the context.
We can see hints at some form of geometry and prototypes from the data on vowels …
Gardenfors notes that …
According to phonetic theory, what determines a vowel are the relations between the basic frequency of the sound and its formants (higher frequencies that are present at the same time). In general, the first two formants F1 and F2 are sufficient to identify a vowel. This means that the co-ordinates of two-dimensional space spanned by F1 and F2 (in relation to a fixed basic pitch F0) can be used as a highly accurate representation of a vowel.
Now, notice what happens when we actually alter the context for a set of objects. The borders between the inferred concepts appear to loosen …
So, it appears that the meaning of concepts can actually be modified by the context. And it also appears that the mind rather effortlessly jumps between these different meanings in a rather natural way, since it can literally happen without our noticing it.
What might be happening with the the candle problem is that offering a reward alters the meaning of the task. A good question to ask, within that context, is perhaps whether or not there are implications for our beliefs about the meanings of concepts within science itself, given that science education & discourse are creating the context for those concepts?
Plasmatic
Re: Online scientific discourse is broken and it can be fixe
its a counter intuitive finding
No, its an anti- reason "trick" with an unspoken background purpose exactly like what you were pointing at earlier.
Hehe, cognitive scientist trying to do epistemology by running an EEG... In a bit Ill show you why there is no substitute for knowledge of the inductive history of philosophical topics.
Edit:
What might be happening with the the candle problem is that offering a reward alters the meaning of the task. A good question to ask, within that context, is perhaps whether or not there are implications for our beliefs about the meanings of concepts within science itself, given that science education & discourse are creating the context for those concepts?
Are you gonna name the irrationalist philosopher who championed this idea?
pln2bz
Re: Online scientific discourse is broken and it can be fixe
No, its an anti- reason "trick" with an unspoken background purpose
I'll be glad to listen to what you have. I do think that you have an uphill battle though, for it's not just creativity researchers who have noticed the existence of subconscious changes in perception about an activity in light of extrinsic rewards. Motivation researchers observe an effect which complements the creativity researchers' findings, but label the effect as the "counterintuitive consequences of extrinsic incentives as the hidden costs of rewards". From Daniel Pink's book, Drive (apologies for the dropped punctuation, btw) …
One of Lepper and Greene's early studies (which they carried out with a third colleague, Robert Nisbett) has become a classic in the field and among the most cited articles in the motivation literature. The three researchers watched a classroom of preschoolers for several days and identified the children who chose to spend their free play time drawing. Then they fashioned an experiment to test the effect of rewarding an activity these children clearly enjoyed.
The researchers divided the children into three groups. The first was the expected-award group. They showed each of these children a Good Player certificate adorned with a blue ribbon and featuring the child's name and asked if the child wanted to draw in order to receive the award. The second group was the unexpected-award group. Researchers asked these children simply if they wanted to draw. If they decided to, when the session ended, the researchers handed each child one of the Good Player certificates. The third group was the no-award group. Researchers asked these children if they wanted to draw, but neither promised them a certificate at the beginning nor gave them one at the end.
Two weeks later, back in the classroom, teachers set out paper and markers during the preschool's free play period while the researchers secretly observed the students. Children previously in the unexpected-award and no-award groups drew just as much, and with the same relish, as they had before the experiment. But children in the first group the ones who'd expected and then received an award showed much less interest and spent much less time drawing. The Sawyer Effect had taken hold. Even two weeks later, those alluring prizes so common in classrooms and cubicles had turned play into work.
To be clear, it wasn't necessarily the rewards themselves that dampened the children's interest. Remember: When children didn't expect a reward, receiving one had little impact on their intrinsic motivation. Only contingent rewards if you do this, then you'll get that had the negative effect. Why? If-then rewards require people to forfeit some of their autonomy. Like the gentlemen driving carriages for money instead of fun, they're no longer fully controlling their lives. And that can spring a hole in the bottom of their motivational bucket, draining an activity of its enjoyment.
Lepper and Greene replicated these results in several subsequent experiments with children. As time went on, other researchers found similar results with adults. Over and over again, they discovered that extrinsic rewards in particular, contingent, expected, if-then rewards snuffed out the third drive.
These insights proved so controversial after all, they called into question a standard practice of most companies and schools that in 1999 Deci and two colleagues reanalyzed nearly three decades of studies on the subject to confirm the findings. Careful consideration of reward effects reported in 128 experiments lead to the conclusion that tangible rewards tend to have a substantially negative effect on intrinsic motivation, they determined. When institutions families, schools, businesses, and athletic teams, for example focus on the short-term and opt for controlling people's behavior, they do considerable long-term damage.
Try to encourage a kid to learn math by paying her for each work-book page she completes and she'll almost certainly become more diligent in the short term and lose interest in math in the long term. Take an industrial designer who loves his work and try to get him to do better by making his pay contingent on a hit product and he'll almost certainly work like a maniac in the short term, but become less interested in his job in the long term. As one leading behavioral science textbook puts it, People use rewards expecting to gain the benefit of increasing another person's motivation and behavior, but in so doing, they often incur the unintentional and hidden cost of undermining that person's intrinsic motivation toward the activity.
This is one of the most robust findings in social science and also one of the most ignored. Despite the work of a few skilled and passionate popularizers in particular, Alfie Kohn, whose prescient 1993 book, Punished by Rewards, lays out a devastating indictment of extrinsic incentives we persist in trying to motivate people this way.
For designers of scientific social networks, the warning that there may be a problem here is actually more important than the level of confidence. That's because the website offers the perfect vehicle for performing what is known as A/B-testing — where slight changes to a web page are tested to see if they increase or maximize an outcome of interest. If-then rewards or reputation systems for creative contributions could easily be tested against prototypical site users to observe if there are any long-term differences.
My current belief is that we needn't go any further than the Nobel Prize to see that offering rewards/recognition as a means of mediating the inherently creative endeavors of scientific discovery and theory-making can distract scientists from an adherence to the values which constitute effective scientific thinking. But, that said, I'm not averse to hearing more on these subjects.
pln2bz
Re: Online scientific discourse is broken and it can be fixe
We Must Change the Way We Think About Paradigm Changes
The hidden curriculum within the graduate science education programs acts as a sort of feedback system which stabilizes scientific theory, in spite of the emergence of promising new ideas in science ...
"Our fundamental problem with unification is that a certain number of ideas have been tried out which all have well-known problems — and string theory is now one of them. But there's a lot of things that haven't been tried … If you start to get to know the subject, you realize the number of people working on the subject … It's a fairly limited community. It's a few thousand people … And … most of them are kind of following the lead of a fairly small number of people. The number of actual different ideas that people are trying out is actually quite small … There's a much larger array of ideas out there which nobody has taken the time to look into because the way the field works … These things are very difficult. You would have to go spend several years of your life doing this, and if no one else is interested in what you're doing (and most likely, whenever you're trying out new ideas, it's not going to work anyways). The way the field is structured, it's kind of very very hard to do that kind of work, because it's likely to damage your career. If you're trying to do it when you're young, you're gonna very well end up not having a job." (YouTube interview of Peter Woit titled "Piling Conjecture Upon Conjecture". At 12:27)
It seems apparent that certain problems in science appear to resist longstanding efforts to solve them. The problem is that we've failed to properly define the problem. If we can accurately define the problem, then we can design a purpose-driven system of discourse which fits together activities that people are already naturally motivated to do. Part of the role of this system should be to point out the blind spots and improvement opportunities which must be collectively overcome in order to make progress on these long-term goals. In other words, we must become experts at critiquing our ourselves.
This is actually something which effective corporations already do better than anybody else. If you carefully look at the most effective corporations in the world, one of the most obvious differences compared with other types of organizations are the types of communications that occur. And one of the most serious challenges for groups of outsider scientists who might be attempting to introduce new ideas into the larger pool of ideas discussed by professional scientists is the lack of financial compensation. After all, if-then rewards are crucial to this process of adaptive change: Without those rewards, we will predictably run into difficulties convincing people to engage in adaptive changes.
This places creative problem-solving in science in the middle of competing forces: Without offering rewards, we lack an ability to convince people to engage in the kinds of adaptive changes necessary to contribute to a solution; yet, if we offer rewards, then there is the risk that those rewards might subtly undermine peoples' motivation to contribute to a solution, or their effectiveness in doing so.
In a general sense, people who develop a fluency in the Electric Universe are (currently) driven at the idea by their own personal interests. However, those interests which propelled them to this interesting point of realization, unfortunately, in no way prepare them to actually act upon what they have learned. What I'd like to propose is that the infrastructure itself can make up the difference.
I've always said that the cause of the trouble is the American graduate school ... You see, when we had graduated first degree, we were independent. We could thumb our noses at the professor. And in fact the best way to get ahead was to do something that all the people didn't agree with ... But, in the graduate school, you all have to learn what the professors are teaching you. And then, those people go out and get jobs and they go to their own graduate school ... You get a few places like CalTech or like Harvard, and they set the fashion for [all the rest] ...Fred Hoyle from the Cosmology Quest video
When a graduate student takes the time to actually learn what the Electric Universe says, and their interest in the subject takes hold, the amount of research and learning necessary to convert that interest into meaningful action such as an experiment is — as things stand — really quite enormous. And if they go to the traditional places where people talk about novel ideas in science, they run into systems of discourse which have not been designed to support this unique endeavor.
The traditional view of paradigm change in science is essentially a passive perspective: The general belief is that if the right technical arguments and evidence are produced, then beliefs in science will over time naturally change on their own. The obstacles which I've presented here suggest that this is not necessarily the case; it may or may not happen, and we are arguably leaving the process to circumstance without any system for intentionally guiding the process from our own perceived problems into directions we conceive as possible solutions. And if our thinking is that we can approach these multi-faceted problems without engaging the whole system — which, truthfully, requires engagement on a wide breadth of topics which diverge far from the base subjects of plasma physics and cosmology — then the solutions we generate using this approach will predictably fail to solve the whole problem. If our preference is to narrow our focus to just those topics which we find entertaining to contemplate, then we should abstain from even proposing solutions to the larger problems of science education and discourse — for these proposed solutions will simply generate other unintended consequences.
Part of my purpose here is to help people to appreciate the sheer breadth of interests which are required in order to properly diagnose the problem and propose solutions; and to propose that if the EU community's interest in science is to be more than just a hobby, it's going to require a discussion which actually matches this breadth.
Plasmatic
Re: Online scientific discourse is broken and it can be fixe
Part of my purpose here is to help people to appreciate the sheer breadth of interests which are required in order to properly diagnose the problem and propose solutions; and to propose that if the EU community's interest in science is to be more than just a hobby, it's going to require a discussion which actually matches this breadth.
Chris, wouldn't it be fun for you and I to demonstrate such a discussion at the "All About Evidence" 2014 conference! We could show what dialogue between contrasting world views looks like by presenting models for evaluating "evidence" in a presentation maybe followed by a talk....
pln2bz
Re: Online scientific discourse is broken and it can be fixe
Interesting idea, but three problems ...
(1) I'm not currently expecting to attend this year's conference.
(2) I'm skeptical of the appropriateness of my own thesis to this year's theme. I think a better approach for the time being would be to engage this problem here on the forum. I'd love to see more people speak up with their thoughts once I've finished summarizing what I have to date.
And (3) I've yet to fully engage the difficult problem of effectively communicating these complex ideas. They do not naturally lend themselves to 17-minute talks, so part of the task at hand is to figure out how to nevertheless express them in such formats. It could take some time to figure this out, and I realize that my current communication style will naturally filter out a lot of interest. Fixing this problem is an important personal goal.
I've been basing my current efforts here on observations of how innovation occurs at a successful Fortune 500 company. I know, based upon detailed discussions with a close friend who works at such a company, that I've yet to fully assimilate all of the lessons necessary to do such innovation. Simple awareness of the need to change is drastically insufficient to actually implement adaptive changes; this stuff is far more difficult than most people realize. While we might argue here on this forum about the hypothetical merits of the self-transformative mindset, people at the more successful corporations will tell you that they already depend upon these principles to do effective innovation in teams.
Unfortunately, this process is so overwhelming that there is little time or effort leftover for working on a talk — which in my own personal experience can take months of preparation.
pln2bz
Re: Online scientific discourse is broken and it can be fixe
How the EU Community Can Redesign Scientific Discourse
Despite all of its apparent flaws, the public has largely internalized the worldview that they can use reputation as an effective metric for formulating beliefs about scientific models. The practice is readily apparent online in forums and in comments attached to scientific press releases, as well as the science journalism which dominates today. Like this ...
Study Finds 97% Consensus on Human-Caused Global Warming in the Peer-Reviewed Literature
A new survey of over 12,000 peer-reviewed climate science papers by our citizen science team at Skeptical Science has found a 97 percent consensus in the peer-reviewed literature that humans are causing global warming.
One of the Electric Universe's most prominent critics offers a rather simplistic reputation-based system for identifying good from bad science:
[…]
So how does Plait's B.S. detector work? Here are some pointers from the pro:
Find out who's making the claim: Certain people have a history of promulgating B.S., and if you come across the latest claim from such people, your skeptical antennae should perk up. "That doesn't mean they're wrong, but it's not a bad place to start," Plait said.
Build a baloney detection kit: Before you get swept up in an artfully crafted argument, apply some scientific principles to the claims. As the late astronomer Carl Sagan said, "Extraordinary claims require extraordinary evidence." Sagan's own baloney detection kit is a good place to start. (The RationalWiki has boiled down his essay to hit the bullet points.)
Find out what other folks are saying: Check to see what your trusted sources have to say about a controversial claim. Of course Plait hopes that Bad Astronomy is one of those trusted sources — but he adds a caveat: "Don't trust me. I make mistakes, too. Pay attention to what other people are saying, weigh the facts, and try not to be biased."
Here are a few of Plait's trusted sources on contentious topics:
[…]
Notice that the advice is offered regardless of the scientific discipline. The more speculative domains like astrophysics and cosmology — where our abilities to observe and experiment are extraordinarily limited — are, in this worldview, treated no differently from the less controversial, more down-to-Earth subjects.
It's peculiar that this practice has become so dominant in our online discourse, given that science's appeal derives largely from its empirical nature. This observation invites us to ask if the reason people rely upon authority so much is because they don't trust their own reasoning skills when it comes to complex scientific models?
It stands to reason that if the public was effectively thinking like a scientist, then they could evaluate the claims themselves, rather than relying upon somebody else to think for them. The predictable rebuttal to this would be that laypeople cannot be expected to reason at the level of a professional specialist scientist. And there are obvious merits to that argument, but we really need to observe caution here: Most people already realize that there is something wrong with the way we teach science today. It naturally follows that if we can figure out what those problems are, and correct for them within our scientific social network's infrastructure, then the public's ability to critically think about scientific models can improve — perhaps even drastically.
I am clearly not the first person to engage this problem. But, I may be the first to approach the problem in this particular way:
Develop a fluency in a promising against-the-mainstream worldview (in our case, this is the Electric Universe).
Use the existing communication systems to attempt to convince people of this novel worldview.
Identify failures of that system to support change in worldview, then use these failures to conceive hypothetical features for a new system of communication.
Identify all of the research topics which are relevant to this endeavor, and meticulously bin out all of the relevant information sources into a system of directories. Make regular backups, as this process should be expected to take years.
Tie these personal revelations from firsthand interactions as best as possible to research from this wide breadth of pertinent topics.
What I'm proposing here is that we reverse-engineer the observed problems of discourse backwards in such a way that those observations can be linked to published research topics like theories of science education, philosophy of science, trends in science education reform, past attempts at visualizing knowledge and argumentation, past efforts at supporting critical and creative problem-solving, and so on. Note that this isn't a comprehensive methodology for creating an actual website, for that would necessarily include focusing upon a very specific target audience and explaining in great detail how this new system will address the needs of that target audience. We are not quite to that point yet.
What I'm suggesting is to prepare our minds with a fabric of relevant information first. And we should use our specialist knowledge in the Electric Universe combined with firsthand experiences attempting to convince others of the EU worldview online, in order to help us to evaluate these longstanding debates in domains like education reform. What this does is make sure that whatever system we might eventually propose will be firmly rooted as a reaction to what is actually happening today within not just the minds of people who are trying to think like a scientist, but also the minds of academics who are perhaps struggling to reform science education.
Now, this is where things are going to start to get a lot more interesting, and even possibly controversial. After all, I've already been doing this for some time now, and I'm here to introduce you to what I've personally discovered. But, my hope is that I can encourage others to follow suit, as I fully expect that different people would come to different conclusions. If I could somehow convince a small community of people to try this out, I am confident that this group could leave an incredibly lasting impact upon the future of science.
pln2bz
Re: Online scientific discourse is broken and it can be fixe
Here's a glimpse at the subjects that I am following. This directory structure has remained stable for about a year now ...
I've found many high-value pockets of information on the web. You would be amazed at what is out there to support this sort of endeavor. There's for instance a guy named Rob Spencer that works at Pfizer who has posted an absolutely incredible collection of resources and presentations on eliciting wisdom from the crowd.
One thing that I've noticed is that philosophers have not stopped writing books on science which are accessible to laypeople just because the public has largely stopped reading philosophy of science books; in fact, it seems to me that the quality of accessible philosophy resources is on a serious uptick.
Based on my research, I can see that people have been trying to figure out ways to visualize argumentation — with little success, apparently — for many decades now. I can also see that concept mapping is a hot research topic in the physics education research community, and can point to some successes. In fact, it's possible to observe a number of important trends in science education research which have thus far failed to break through into actual educational techniques within our university programs. The larger explanatory framework offered by Jeff Schmidt can help us to understand why that is.
Machine learning — a subset of artificial intelligence — started out as a marginally relevant topic for this project, but then one day I was tipped off in a physorg article to this idea of topic modeling. Once I learned that important keyword, I was able to track down an incredible amount of information on this very important subject. This is a recurring occurrence actually: the discovery of a particular keyword can open up all sorts of previously unknown doors to relevant academic research.
The solution to this problem is at the intersection of all of these divergent areas of research, which suggests that our approach should be to research them all concurrently. There's not too much sense to hyper-focusing upon just one aspect of the investigation, with the intention that we'll one day get to that other stuff, in some sort of a sequential research process. What we need to do is to create a wide breadth of knowledge designed specifically to address this very specific problem. The point is not to spread ourselves out so thin that it undermines our ability to focus; the point is to make our decisions about what to focus upon more meaningful — and to avoid becoming too bogged down in just those topics which we find personally interesting.
I've to-date collected a bit over 6 TB of relevant sources, when the science-related materials are included. If there are others out there who share my interest, I can flip a switch on the Mac Server. And we can together get serious about this. There's no need to start from scratch here.
pln2bz
Re: Online scientific discourse is broken and it can be fixe
It is absolutely correct for onlookers to call for increased skepticism and clearer thinking in science writing. I've sometimes heard it said, with a certain amount of condescension, that this or that field of science "needs its popularizers." But what science really needs is greater enthusiasm for those people who are willing to invest the time to try to sort the truth from hype and bring that to the public. Academic science does far too little to encourage such voices.
pln2bz
Re: Online scientific discourse is broken and it can be fixe
The Unseen Danger of Dogma
There are a handful of amazing stories which help us to define the problems that we face in science and science education at this current moment. Hearing these stories can have a lifelong effect upon a person's views of the subject. David Talbott's biographical piece in Edge Magazine dedicated to Hannes Alfven is one such story, and part of what sets it apart from other stories is the apparent decision to not to teach this important story to modern-day undergraduate and graduate students:
… But the critical turn in this story, the part almost never told within the community of astronomers and astrophysicists, is that Alfvén came to realize he had been mistaken. Ironically—and to his credit—Alfvén used the occasion of his acceptance speech for the Nobel Prize to plead with scientists to ignore his earlier work ...
There are apparently a number of these important stories which seem to have failed to manifest into lessons in the conventional curricula. The story told by Gerrit Verschuur of the advent of radio astronomy in The Invisible Universe is yet another one. It suggests the inherent risk of leaving scientific thinking to just the experts. When it was first discovered by an outsider amateur that radio waves were originating from the sky itself,
"Radio engineers didn't care where the radio waves came from, and the astronomers '… could not dream up any rational way by which the radio waves could be generated, and since they didn't know of a process, the whole affair was (considered by them) at best a mistake and at worst a hoax.
Even if today we tend to fear most the layperson's tendency to mangle expertise, the refusal to even consider radio waves coming at us from space reminds us that even relatively modern scientists can apparently "know too much". The threat inherent to dogma is no less concerning than the layperson's tendency to mangle theory. It's simply harder to see. Together, these two stories can help us to see that there can be unavoidable risk in every direction in science. The critical thinkers of the world are simply caught between inaccurate non-professionals and highly-accurate-but-wrong professionals.
The Paradigm Shift in Science Education that Is Already Underway
The physics education reform movement has its own such big story. This intriguing article is required reading for anybody who has an interest in science education, because it suggests a promising direction for us to focus on …
What Drives College Students to Learn?
Back in the late 1970s a colleague came to David Hestenes with a problem.
The two of them were physics professors at Arizona State University. Hestenes was teaching mostly graduate students, but his colleague was teaching introductory physics, and the students in his classes were not doing well. Semester after semester, the class average on his exams never got above about 40 percent.
"And I noted that the reason for that was that his examination questions were mostly qualitative, requiring understanding of the concepts," says Hestenes.
Most professors didn't test for this kind of understanding; students just had to solve problems to pass the exams.
This observation prompted a series of conversations between Hestenes and his colleague about the difference between being able to solve problems and really understanding the concepts behind those problems. They had a sneaking suspicion students were just learning the problem-solving part and never really getting the concepts.
This is the under appreciated story of the creation of the test known as the force concept inventory test — or FCI, for short. The FCI is a multiple choice test which is fundamentally designed to test for conceptual comprehension, rather than problem-solving skills. It's a unique test in that it intentionally avoids the use of any jargon. Researchers give the test both before and after a semester-long course, which permits them to diagnose the change in conceptual comprehension. This new metric is having a transformational effect upon how all of science is taught today at the university level even though the technique would appear to be limited in scope to introductory materials which can be reduced to natural language explanations. For a detailed summary of the history and a discussion of the debate surrounding the FCI, see Richard Hake's "The Impact of Concept Inventories on Physics Education and Its Relevance for Engineering Education".
A brief glimpse at the table of contents ...
I review the:
Before Concept Inventory (BCI) dark ages of post-secondary introductory physics education;
1985 advent of the first physics CI, the Mechanics Diagnostic (MD) by Halloun & Hestenes (HH);
1987-90 early research use of the (MD) by HH, Hake, and Crouch & Mazur;
1992 Force Concept Inventory (FCI), successor to the MD, and the Factor Analysis Debate (1995);
1995 revision of the FCI by Halloun, Hake, Mosca, and Hestenes;
1998 meta-analysis of FCI/MD results on 62 introductory physics courses (N = 6542) showing about a two-standard-deviation superiority in average normalized gains <g> for "interactive engagement" over traditional passive-student lecture courses by Hake and subsequent confirmation by about 25 other physics education research studies.
I then indicate:
fourteen hard lessons from the physics education reform effort;
suggestions for the administration and reporting of CI's;
listings of CI's, including those for physics and engineering; and comment that:
for physics education the road to reform has been all uphill;
the glacial inertia of the educational system, though not well understood, appears to be typical of the slow Diffusion of Innovations [Rogers (2003)] in human society;
there are at least "Eleven Barriers to Change in Higher Education";
but, even so, for physics education, Rogers' "early adopters" of reform have now appeared at Harvard, North Carolina State University, MIT, the Univ. of Colorado, California Polytechnic at San Luis Obispo, and the Univ. of British Columbia, possibly presaging a Rogers "take off" for physics education reform, about two decades ACI (After Concept Inventory).
I conclude that:
CI's can stimulate reform, but judging from the results in physics it may take about two decades before even early adopters become evident;
there are at least seven reasons why the rate of adoption of reforms may be greater in engineering education than in physics education.
In an Appendix I respond to criticisms of the FCI and the average normalized gain <g>
The emphasis there is mine, and meant to indicate that the FCI is viewed by some as not just a new metric, but also a defining moment for the physics education reform movement. And this invites us to ask a question as we learn more: If the FCI is a new metric which claims to measure an ongoing problem in how we train people to think about science, then isn't that problem still a part of all of us who were formerly taught almost exclusively through problem sets and lectures? Aren't the ramifications of that failure now culturally encoded into the very way in which we think about scientific theories today?
Consider the introduction to a seminal paper on this topic titled simply, "Force Concept Inventory":
Every student begins physics with a well-established system of commonsense beliefs about how the physical world works derived from years of personal experience. Over the last decade, physics education research has established that these beliefs play a dominant role in introductory physics. Instruction that does not take them into account is almost totally ineffective, at least for the majority of students.
Specifically, it has been established that (1) commonsense beliefs about motion and force are incompatible with Newtonian concepts in most respects, (2) conventional physics instruction produces little change in these beliefs, and (3) this result is independent of the instructor and the mode of instruction. The implications could not be more serious. Since the students have evidently not learned the most basic Newtonian concepts, they must have failed to comprehend most of the material in the course. They have been forced to cope with the subject by rote memorization of isolated fragments and by carrying out meaningless tasks. No wonder so many are repelled! The few who are successful have become so by their own devices, the course and the teacher having supplied only the opportunity and perhaps inspiration.
One way to think about this is in terms of languages. As is commonly done, we might characterize mathematics as the language of nature. That's fair enough, but does that necessarily mean that people always use mathematics to reason about the universe? No, of course not. Most people who are engaging the subject of science are actually spending the bulk of their time thinking in terms of science's concepts. The quantitative aspects of problem-solving help us to characterize these conceptual relationships. But, the concepts and their properties are atomic to human thought, and mathematics is ill-equipped to act as a substitute infrastructure for this type of thought. Mathematics very well may be the language of nature. But, when it comes to decisions about which ideas to focus upon in science, concepts will prove to be the more fundamental language. What the physics education research community appears to be telling us is that science education which focuses more upon recipe-based problem-solving than conceptual comprehension will generally generate thinkers who lack the language necessary to alter their own scientific worldviews.
People, they are talking about all of us. Nobody is immune on these points. We've all been trained in more-or-less the same way here.
I've got more good materials on the FCI coming up. We're not quite done here yet ...
pln2bz
Re: Online scientific discourse is broken and it can be fixe
"There is nothing more difficult to plan, more doubtful of success, nor more dangerous to manage than the creation of a new order of things … Whenever his enemies have the ability to attack the innovator they do so with the passion of partisans while the other defend him sluggishly, so that the innovator and his party alike are vulnerable."
