Total Posts:25|Showing Posts:1-25
Jump to topic:

Scientific peer-review system

ithink-ithink
Posts: 11
Add as Friend
Challenge to a Debate
Send a Message
11/3/2015 5:06:11 PM
Posted: 1 year ago
Is the peer-review system really what we should be using in science? I am a low-level scientist(MSc), so I just wanted to know what some others may have to say about this.

I know the basis behind why it is used; that the people best certified to validate an incoming scientific paper is those scientist in the relevant field.
But what happens when a paper comes about, that would potentially cause a paradigm shift. If the shift is too far from conventional scientific wisdom, isn't there a much higher probability that the paper would be rejected purely on the basis that the scientists in the field (doing the peer review) can't reconcile their life's work/understanding with the paper.

I gather that the old system through which papers were accepted into journals consisted of a multi-disciplinary panel which judged papers on their fluency, logic, quality of experimental evidence etc. Isn't this system better? People's papers would be judged on their own standalone quality, thereby systematically discouraging consensus and scholasticism, as opposed to the peer-review system which seems to systematically encourage it.

Just wanted to hear people's thoughts.
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
11/3/2015 9:31:42 PM
Posted: 1 year ago
At 11/3/2015 5:06:11 PM, ithink-ithink wrote:
I know the basis behind why it is used; that the people best certified to validate an incoming scientific paper is those scientist in the relevant field.
But what happens when a paper comes about, that would potentially cause a paradigm shift. If the shift is too far from conventional scientific wisdom, isn't there a much higher probability that the paper would be rejected purely on the basis that the scientists in the field (doing the peer review) can't reconcile their life's work/understanding with the paper.

That's a great question, IT^2... It's true that some valid and ultimately successful ideas were dismissed early. (The "Big Bang" Theory, for example.) But the reason those ideas recover relates to mutual evidentiary accountability, and here's how that works:

If I submit to you a new conjecture, I'm accountable to you for its evidentiary falsification. In other words, I need to be able to say under what observable conditions my conjecture should be discarded -- and I'm accountable for finding the simplest observable conditions under which falsification would occur, not to 'lift the bar' to make the conditions needlessly hard. :)

But conversely, if you favour an existing idea, then you and everyone who favours it are accountable for finding the simplest observable conditions under which falsification would occur for it too.

That creates a dialogue of mutual evidentiary accountability between us. Building respect for the evidence more than admiration of our own ideas, it transcends culture and prejudice, so that, as cosmologist and science communicator Neil deGrasse Tyson put it:

Any time scientists disagree, it's because we have insufficient data. Then we can agree on what kind of data to get; we get the data; and the data solves the problem. Either I'm right, or you're right, or we're both wrong. And we move on. That kind of conflict resolution does not exist in politics or religion.

Or as cosmologist Carl Sagan put it:

In science it often happens that scientists say, 'You know that's a really good argument; my position is mistaken,' and then they would actually change their minds and you never hear that old view from them again. They really do it. It doesn't happen as often as it should, because scientists are human and change is sometimes painful. But it happens every day. I cannot recall the last time something like that happened in politics or religion.

That ethic of mutual evidentiary accountability, coupled with a commitment to best practice methods in identifying and removing imprecision, is really the soul of scientific advancement.

With that said, some social effects can occur. For example, if the editor of a journal favours a theory or method, he or she can select (whether consciously or not) only reviewers known to favour the theory or method too. If an institution has staked its reputation on a theory or method, it may hire only supporters.

So diversity is important... it's key that there are many quality journals, many quality institutions, many methods, many theories, many predictions and that they vie with one another to differentiate themselves in precision and insight.

I hope that may help. :)
ithink-ithink
Posts: 11
Add as Friend
Challenge to a Debate
Send a Message
11/4/2015 5:13:06 PM
Posted: 1 year ago
*sigh*

Wrote a response and my browser bombed.

Ok, bullet point version, with less reasoning;

- I think that the peer-review system is quite robust, but I do have doubts about how it would be able to handle paradigm shifts.
- It relies on scientists in a field being completely true to the ideals of science, even if they could lose their careers (if the shift is extreme enough). Even though I agree that scientists are far more adaptable than politics/society, there is a limit.
- Evidentiary falsification could exist without the peer-review system, so the peer review system isn't necessarily intrinsically linked to that, surely?
- Considering that, shouldn't we create/be using a system which would negate or at least decrease the likelihood than conventional wisdom may repress novel, potentially revolutionary advances in science?
- Isnt it better to have a system which systematically favours novel ideas, rather than systematically favouring conventional thought?
- Technically, the peer-review system may have already repressed some ideas that may have changed entire fields; we wouldn't exactly know. I just think it is a bit of a worry.

- What you said about differing journals/institutions does help though. Im guessing that that would mean there are a few institutions which pride themselves on searching for more novel approaches. Accessibility to someone with a novel idea might be a discouraging factor though.
ithink-ithink
Posts: 11
Add as Friend
Challenge to a Debate
Send a Message
11/4/2015 5:34:36 PM
Posted: 1 year ago
I suppose the peer review system also favours fields with far more evidence. By that I mean, biological/mechanical phenomena.

So the problem would really occur most frequently in highly theoretical fields like mathematics and physics (pure).
kp98
Posts: 729
Add as Friend
Challenge to a Debate
Send a Message
11/4/2015 6:52:21 PM
Posted: 1 year ago
What's the alternative?

There has to be some way of distinguishing between good solid scientific work and crackpots claiming to have invented perpetual motion.

Its a good thing that 'paradigm shifts' are hard to achieve, because real paradigm shifts are rare - usually like 'cold fusion' they are less to do with paradigm shifts than shoddy science or uncalibrated measuring devices.

There is a good reason established science is conservative - the existing paradigm is tested thousands of times every day directly and even more times indirectly. A new paradigm has to not only be good for some single phenomenon we don't know but also explain what we do know. If it can do that, then its truth will allow it to be presented well enough to be peer-reviewed.

Peer review does allow real (but rare) paradigm shifts to be separated from (far more common) crackpot theories. I am not sure any other system would do as well.
JMcKinley
Posts: 314
Add as Friend
Challenge to a Debate
Send a Message
11/4/2015 7:37:22 PM
Posted: 1 year ago
You guys are misunderstanding what the "peers" are reviewing. The purpose of peer review isn't for them to confirm or deny the hypothesis. That is the job of the experimental data.

The reviewers are analyzing your experiment methodology, and your data analysis to ensure that you haven't made a mistake or completed a faulty experiment. They can't give a thumbs down just because they don't like the idea. They have to show fault in your math or your method. If they can't demonstrate that fault, then they can't reject your paper.

Now obviously the system doesn't always work as designed, just as any human system, but I do think that its is sufficiently effective to maintain steady scientific progress as can be demonstrated by our recent history. In short, I'd say that its not perfect, but it is effective.
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
11/4/2015 11:49:07 PM
Posted: 1 year ago
At 11/4/2015 7:37:22 PM, JMcKinley wrote:
You guys are misunderstanding what the "peers" are reviewing. The purpose of peer review isn't for them to confirm or deny the hypothesis. That is the job of the experimental data.
Yes, exactly.

The reviewers are analyzing your experiment methodology, and your data analysis to ensure that you haven't made a mistake or completed a faulty experiment.

That's definitely part of it, JM, but there's more than that in an evaluation. We can think of a research paradigm as being part ontology (what is known to exist and how to classify it), part epistemology (what are truth and knowledge, and how to recognise their robustness), and part methodology (what tools, techniques, and standards are accepted.) In the sciences, these come together to form a 'virtuous circle' of objective accountability: each can hold the others to account.

So methodology is itself subject to improvement driven by ontology or epistemology. Consider the change in biological methods driven by a shift from Linnaean taxonomy [https://en.wikipedia.org...] to cladistics [https://en.wikipedia.org...]: that's ontology driving method. Or consider the adoption of blind- and double-blind experiments: in psychology -- that's epistemology driving method.

And further, one can (and should) peer review conjectures, hypotheses, models, predictions and research paradigms, and not just experimental results. The purpose of peer review is to reduce the risk of imprecision -- of conception, planning, method, recording, interpretation, evaluation and reporting. So these quality assurance steps can happen right throughout the process, and the earlier and more frequently it's done, the greater the potential benefit.

Science can be thought of as the engineering of robust natural knowledge. And as engineers know, if you want quality, you have to build it in to ever step of the process.

Drawing as it does on best practice expertise, peer-review is a key quality planning and assurance step in knowledge engineering.
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
11/5/2015 12:30:52 AM
Posted: 1 year ago
At 11/4/2015 5:13:06 PM, ithink-ithink wrote:
- I think that the peer-review system is quite robust, but I do have doubts about how it would be able to handle paradigm shifts.
- It relies on scientists in a field being completely true to the ideals of science, even if they could lose their careers (if the shift is extreme enough).

Ignoring the fairness to young scientists with new ideas for a moment, you want high levels of skepticism and scrutiny in dealing with innovation. Science can produce far more inaccuracy trusting things that don't work than distrusting things that do. :)

But paradigm shifts tend to happen not simply from the vision of an individual scientist, but from growing intellectual pressure in the discipline as a whole. When an accepted paradigm cannot account for key observations no matter how hard you try, nobody can publish anything that matters, and so everyone gets frustrated. It only takes one or two generations of scientists to start trying new ontologies, new models, and questioning epistemological assumptions -- this has happened repeatedly in every discipline.

Initially those new ideas tend to be conjectural, and conjectures get beaten up pretty badly in science... but that bludgeoning represents a dialogue from which hypotheses emerge -- and many of those fail, but then some produce promising results, and that leads to more conjecture, hypothecation, experiment and consolidation into coherent theory. Or sometimes, there's an extant theory that just hasn't been applied correctly. (Relativity drew heavily on some half-right work by Lorentz, for example.)

So much like evolution, social pressures in science tend to work against rapid innovation in the short term, but toward robust innovation in the long term. (Except when a new field opens up -- then things can take off for a while, until everything begins to consolidate.)

- Evidentiary falsification could exist without the peer-review system, so the peer review system isn't necessarily intrinsically linked to that, surely?
That's true. However, there can be a lot of subjectivity in ontology and epistemology. To see how bad this can get, consider Isaac Newton. A fine theoretician and a decent empiricist, he did all his decent science by about age 27. After that he wasted his life on alchemy, simply because the ontological and epistemological foundations of chemistry hadn't been established yet -- and sadly, he wasn't the guy to establish them.

- Considering that, shouldn't we create/be using a system which would negate or at least decrease the likelihood than conventional wisdom may repress novel, potentially revolutionary advances in science?
Such a system is already in place in science. Any scientist is free (at his or her own risk) to defy conventional paradigms just to see what may come of that. (Theoreticians often do this, since theoretical science is somewhat entrepreneurial: you try and anticipate the empiricism to guess what the problems might be and how they might be resolved.) Mathematics, for example, is full of theoretical structures for which there are no or few empirical uses -- and it has been argued that superstring theory is such a construction too.

However, you can't defy the underlying scientific ethics -- which are to do with accountability, transparency, falsification and honesty. When you do that, you're putting a tinfoil hat firmly on your head, and frolicking off into the realm of pseudoscience.

Scientists tend not to produce pseudoscience by accident. They either deliberately abandon scientific ethics and depart the scientific community (Deepak Chopra as an example), or get pulled back into intellectual accountability by demands for falsifiability (superstring theory being an example.)

- What you said about differing journals/institutions does help though. Im guessing that that would mean there are a few institutions which pride themselves on searching for more novel approaches. Accessibility to someone with a novel idea might be a discouraging factor though.

I think it depends on the science. Some science is big and capital-intensive (think Mars-shots, and CERN's Large Hadron Collider.) The experiments and observations conducted there tend to be expensive, so you want to be confident it'll have a chance of showing real results. On the other hand, Big Discoveries help justify the expense, so the institutions managing those facilities are constantly trading off risk and reward.

But when science is inexpensive (Virtual Reality, for example), pretty much anyone can start a lab and explore. Capital equipment for such an initiative might be only a few tens of thousands of dollars, and easy to rustle up -- the rest is salaries and on-costs.

Back on the question of fairness to individual scientists -- sadly, like evolution, it's not. There are always more questions than answers, so it's not hard to move most fields forward somewhat. However intelligence and hard work don't bestow grand destiny in science -- that's something of a popular myth. The few really landmark results that are around to be discovered depend on you being in the right field at the right time and (probably) working in the right institution or at least, being in contact with the right people.

Most big results in science are a race between multiple parties aiming at the same publication -- which means that lots of people can see the same potential. That in turn provides some assurance that important things aren't likely to be missed.
ithink-ithink
Posts: 11
Add as Friend
Challenge to a Debate
Send a Message
11/5/2015 5:18:53 PM
Posted: 1 year ago
OK I think I'm getting where you're coming from a little more. But I still have misgivings. Perhaps that is because I still don't know all the different platforms scientists are able to vocalise their views, but Ill talk about it in quote.

At 11/5/2015 12:30:52 AM, RuvDraba wrote:

Ignoring the fairness to young scientists with new ideas for a moment, you want high levels of skepticism and scrutiny in dealing with innovation. Science can produce far more inaccuracy trusting things that don't work than distrusting things that do. :)

Well, I agree with you in most cases. I think my issue here is the "trust". I'm not saying that one should go and found a whole new branch of a subject on barely passable evidence. But surely it would lend itself to science to have a recognised platform where people can put out theories, so that if there is someone else, who gets exposed to that theory gets 'inspired' (I dont really like saying it like that but lets go with it) by the theory, and manages to form an altered, more testable theory.
I suppose conjecture would be the right term to use there.

Is there such a platform? Perhaps I am just not aware of them.

But paradigm shifts tend to happen not simply from the vision of an individual scientist, but from growing intellectual pressure in the discipline as a whole. When an accepted paradigm cannot account for key observations no matter how hard you try, nobody can publish anything that matters, and so everyone gets frustrated. It only takes one or two generations of scientists to start trying new ontologies, new models, and questioning epistemological assumptions -- this has happened repeatedly in every discipline.

I think Im too young/havent read enough on the history of science over the last 100 years to have seen this :P
My father (Geophysicist/Engineer) is of the opinion that physics has been stuck in this position ever since Einstein. I'm sure the peer-review system was not in the same state as it is now, if it was even around at all. Correct me if I'm wrong? If I'm wrong, never mind. But if I'm right, then are you of the opinion that Einsteins theory would have been able to stand up to the scrutiny of the peer-review system today?

Initially those new ideas tend to be conjectural, and conjectures get beaten up pretty badly in science... but that bludgeoning represents a dialogue from which hypotheses emerge -- and many of those fail, but then some produce promising results, and that leads to more conjecture, hypothecation, experiment and consolidation into coherent theory. Or sometimes, there's an extant theory that just hasn't been applied correctly. (Relativity drew heavily on some half-right work by Lorentz, for example.)

Yes, that's what I meant from before. Where are these platforms? Are they only inside finite entities, or are they published so that they can reach a wider community? I only mention this disparity because my assumption would be that the more scientists who are able to have access to that information would increase the probability that something innovative comes out of it.

So much like evolution, social pressures in science tend to work against rapid innovation in the short term, but toward robust innovation in the long term. (Except when a new field opens up -- then things can take off for a while, until everything begins to consolidate.)

Haha I never thought of it as an emergent-type system. That is probably the best argument I've heard for the peer-review system. My only argument would be that social aspects are acting as too much of a selective pressure. But yes, I suppose you're right. Eventually the strongest theory would win.

That's true. However, there can be a lot of subjectivity in ontology and epistemology.

True, but science was rather a small field at that point. Peer-review, even if it was in place at the time of Isaac Newton, probably wouldn't have stopped that. Surely the - how do I say it, I don't have the right vocabulary like you do, sorry if this is unclear - 'reason' for that was due to a lack of alternative/surrounding theories which Newton could have referred to, not necessarily directly to his work, but at least would be able to put him in the correct frame of mind. What I mean, relating to now, is that we have a lot of scientific 'backdrop' to put us on the right path; it isn't necessary to actually have scientists in your field critiquing your work from a perspective of their work for you to know when your own theory is implausible. Does the fact that a potential theory doesn't 'fit in' with conventional thought degrade the worth of that work to science as a whole?

Such a system is already in place in science. Any scientist is free (at his or her own risk) to defy conventional paradigms just to see what may come of that. (Theoreticians often do this, since theoretical science is somewhat entrepreneurial: you try and anticipate the empiricism to guess what the problems might be and how they might be resolved.) Mathematics, for example, is full of theoretical structures for which there are no or few empirical uses -- and it has been argued that superstring theory is such a construction too.

Yes, I have noticed that there is a little more freedom in those fields, but I feel like it is though those fields in particular would lend themselves to use of a different system. In fact, perhaps only those fields; from my observations, peer-review works very well in my field (bchem/chem).

However, you can't defy the underlying scientific ethics -- which are to do with accountability, transparency, falsification and honesty. When you do that, you're putting a tinfoil hat firmly on your head, and frolicking off into the realm of pseudoscience.

I would never intend to throw those away; I just don't think that they are inherently tied to only peer-review.

Back on the question of fairness to individual scientists -- sadly, like evolution, it's not. There are always more questions than answers, so it's not hard to move most fields forward somewhat. However intelligence and hard work don't bestow grand destiny in science -- that's something of a popular myth. The few really landmark results that are around to be discovered depend on you being in the right field at the right time and (probably) working in the right institution or at least, being in contact with the right people.


Perhaps I still have too much of a young naiive mentality; get everything done NOW :P
If I were looking at it from a perspective of 'the good of science' as a whole, the peer-review system is very sturdy.
ithink-ithink
Posts: 11
Add as Friend
Challenge to a Debate
Send a Message
11/5/2015 5:24:38 PM
Posted: 1 year ago
At 11/4/2015 7:37:22 PM, JMcKinley wrote:
You guys are misunderstanding what the "peers" are reviewing. The purpose of peer review isn't for them to confirm or deny the hypothesis. That is the job of the experimental data.

The reviewers are analyzing your experiment methodology, and your data analysis to ensure that you haven't made a mistake or completed a faulty experiment. They can't give a thumbs down just because they don't like the idea. They have to show fault in your math or your method. If they can't demonstrate that fault, then they can't reject your paper.

Now obviously the system doesn't always work as designed, just as any human system, but I do think that its is sufficiently effective to maintain steady scientific progress as can be demonstrated by our recent history. In short, I'd say that its not perfect, but it is effective.

No that isn't true at all. While a paper will be rejected for 'shoddy science' (false data, shoddy instrumentation, false reasoning), it can also be rejected for a plethora of other reasons. And yes, sometimes, because it is non-conventional, and so there aren't enough peers whose work agrees with your work, and so it doesn't reach publication, even if it is a scientifically sound paper. As RuvDraba pointed out to me though, there is some flexibility between journals.
ithink-ithink
Posts: 11
Add as Friend
Challenge to a Debate
Send a Message
11/5/2015 5:37:33 PM
Posted: 1 year ago
At 11/4/2015 6:52:21 PM, kp98 wrote:
What's the alternative?

I don't have one personally. I am just asking what other people's opinions are on the matter. And whether it would lend itself to science to create a new one considering the the possible shortfalls of our current one.

There has to be some way of distinguishing between good solid scientific work and crackpots claiming to have invented perpetual motion.

Its a good thing that 'paradigm shifts' are hard to achieve, because real paradigm shifts are rare - usually like 'cold fusion' they are less to do with paradigm shifts than shoddy science or uncalibrated measuring devices.

I understand that they are rare. But considering their rarity, I would say that a system which is able to get that idea circulated quickly, would be better. Surely?
I do understand what problems a system like that would face (wild allegations with no data, poorly executed research, pseudo-science); but surely if we designed a system, we could knowledgeably counter that.

There is a good reason established science is conservative - the existing paradigm is tested thousands of times every day directly and even more times indirectly. A new paradigm has to not only be good for some single phenomenon we don't know but also explain what we do know. If it can do that, then its truth will allow it to be presented well enough to be peer-reviewed.

Hm, how can I say this. The problem I am referring to, I think could be explained using an example.
Say in physics; someone suddenly unifies all the forces (big jump, I know :P); entire careers in string theory, potentially dark matter etc. will be lost at the drop of a hat. So a new incoming theory may face a lot of animosity from the scientific community because of the repercussions of that paper. More likely, it will take years before that theory would be acceptable, as people have time to accustom. themselves to it.
This is the problem I mean.

Peer review does allow real (but rare) paradigm shifts to be separated from (far more common) crackpot theories. I am not sure any other system would do as well.
-It would depend on the system. I wouldn't say it's impossible to have another system which could do it, while also addressing the problems of conventional wisdom. No I don't have a design for another system, I just have misgivings about peer-review.

RuvDraba: However, you can't defy the underlying scientific ethics -- which are to do with accountability, transparency, falsification and honesty. When you do that, you're putting a tinfoil hat firmly on your head, and frolicking off into the realm of pseudoscience.

Me :D : I would never intend to throw those away; I just don't think that they are inherently tied to only peer-review.
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
11/5/2015 7:58:08 PM
Posted: 1 year ago
At 11/5/2015 5:18:53 PM, ithink-ithink wrote:
At 11/5/2015 12:30:52 AM, RuvDraba wrote:
I think my issue here is the "trust". [...] Surely it would lend itself to science to have a recognised platform where people can put out theories,
Any scientist can publish unreviewed material. Monographs can be published unreviewed, so can technical notes (sometimes published on institutional web-sites), and so can blogs. Papers in submission can also be published on institutional web-sites before the review is complete. Some journals also don't peer review, or review for other criteria than scientific rigour. And anyone can make press-releases.

But this is dangerous practice whether your work is good, half-good or bunkum. For reasons of rigour, scientists may choose to not cite unreviewed material in their own work, and this can affect measures of impact (such as citation count.) Institution productivity measures may also suffer, and that can affect research funding. Your work may be exposed to media interest before it's exposed to expert scientific scrutiny (and if you'd like to see how bad that get, you could ask Pons and Fleischmann [http://undsci.berkeley.edu...].) Quality can suffer, and so can collaboration opportunities. And let's not forget patents.

Finally, outside the peer review process, scientific publication is a mess. It's full of pseudoscientists 'proving' telekinesis, homeopathy, astral travel, creationism, the existence of God and the soul using flawed ontologies, epistemologies and methods that peer review was set up to prevent. All those authors are complaining that science 'won't accept their revolutionary paradigm-shift' too -- which points out the very problem of shifting paradigm without conserving rigour. So you'll publish in some very shady company.

Moreover, conjectures can already be published in peer-reviewed organs, and this is something theoreticians often do. The evaluation criteria for a conjecture are different to that for an experiment, but might be loosely listed as:
* Would this solve a recognised problem;
* Does it account for all the evidence;
* Is it falsifiable in principle;
* Are all its implications defensible;
* Is it likely to lead to a testable hypothesis;
* Does it list possible mechanisms;
* Would it be testable under existing or emerging methods;
* Are there early indicators for its likelihood of success;
* How does it compare to other conjectures?

But paradigm shifts tend to happen not simply from the vision of an individual scientist, but from growing intellectual pressure in the discipline as a whole.
I think I'm too young/haven't read enough on the history of science over the last 100 years to have seen this :P
If you dig into the landmark results of the last century and a half -- e.g, germ theory, inheritance, evolution, radioactivity, relativity, continental drift, big bang, DNA structure... virtually all had multiple people working independently in the area, often with similar ideas. Journalism and popular imagination tend to iconify the Great Man theory and conflate methodical investigation with inspired vision, but if Pasteur, Mendel, Darwin, the Curies and Einstein had all gone down with Buddy Holly, it's hard to see that science would have failed to produce those results.

Are you of the opinion that Einsteins theory would have been able to stand up to the scrutiny of the peer-review system today?
For sure. Firstly, there was already a known problem with absolute frames, highlighted by the Michelson-Morley experiment, and the whole notion of aether had nearly collapsed under its own weight. Mathematician David Hilbert actually scooped Einstein in publication submission on the same idea by five days, but didn't press priority. Poincare also had much the same idea. [http://www.nobelprize.org...] Special relativity wasn't a bolt from the blue for anyone following the science -- which is not the same as being uncontroversial.

Initially those new ideas tend to be conjectural, and conjectures get beaten up pretty badly in science... but that bludgeoning represents a dialogue from which hypotheses emerge -- and many of those fail, but then some produce promising results, and that leads to more conjecture, hypothecation, experiment and consolidation into coherent theory.
Yes, that's what I meant from before. Where are these platforms?
From personal experience, conferences are a great place to present conjectures in a critical-but-friendly atmosphere, and conference dinners are a great place to bring out the whackier ideas, somewhere between dessert and coffee. Plus, you can share whatever nutty ideas you have with whomever you work with -- or people with similar interests via email, Skype or whatever you want.

Journal-published science is formal and conservative, but in my experience, for all its apparent pedantry, institutional science is a bunch of iconoclasts, anarchists and ratbags united by a common car-park. :)

So much like evolution, social pressures in science tend to work against rapid innovation in the short term, but toward robust innovation in the long term. (Except when a new field opens up -- then things can take off for a while, until everything begins to consolidate.)
Haha I never thought of it as an emergent-type system. That is probably the best argument I've heard for the peer-review system. My only argument would be that social aspects are acting as too much of a selective pressure.
Yes they are, but the social effects are more complicated and far-reaching than simply peer review. We could talk about that more if you're interested, IT^2, but it'll ramble and we might want to start a new thread.

There can be a lot of subjectivity in ontology and epistemology.
True, but science was rather a small field at that point. Peer-review, even if it was in place at the time of Isaac Newton, probably wouldn't have stopped that. Surely the - how do I say it, I don't have the right vocabulary like you do, sorry if this is unclear - 'reason' for that was due to a lack of alternative/surrounding theories which Newton could have referred to, not necessarily directly to his work, but at least would be able to put him in the correct frame of mind. What I mean, relating to now, is that we have a lot of scientific 'backdrop' to put us on the right path
That's exactly right. To rephrase: the stronger our ontology, epistemology and methodology, the more robust the paradigm on which to hang new ideas, and the harder to hang bad ideas. This persists even if the paradigm itself is replaced, since any new paradigm has to be at least as good at falsifying bad ideas as the old one.

You're likely aware that science doesn't proceed by identifying truth; it progresses by eliminating inaccuracy and imprecision. And because it's a continuous improvement process driven by best practice, once a bad idea is eliminated, it can't return so long as the ethics are upheld, and everyone has access to the same information. And the product of that filtering is codified in those three components: ontology (what exists and how we classify it), epistemology (what knowledge and falsehood look like), and methodology (how to validate and verify our guesses.)

.../ctd
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
11/5/2015 8:07:19 PM
Posted: 1 year ago
At 11/5/2015 5:18:53 PM, ithink-ithink wrote:
At 11/5/2015 12:30:52 AM, RuvDraba wrote:
Does the fact that a potential theory doesn't 'fit in' with conventional thought degrade the worth of that work to science as a whole?
It depends on how it doesn't fit. :) New science can't ignore the successful predictions of old science, previous reasons for rejecting bad ideas, or known issues with ontology, epistemology and methodology. So it has to integrate the lessons learned, even if it offers new models. If it doesn't, it enters the tinfoil hat territory of pseudoscience.

Such a system is already in place in science. Any scientist is free (at his or her own risk) to defy conventional paradigms just to see what may come of that. (Theoreticians often do this, since theoretical science is somewhat entrepreneurial: you try and anticipate the empiricism to guess what the problems might be and how they might be resolved.) Mathematics, for example, is full of theoretical structures for which there are no or few empirical uses -- and it has been argued that superstring theory is such a construction too.
Yes, I have noticed that there is a little more freedom in those fields, but I feel like it is though those fields in particular would lend themselves to use of a different system. In fact, perhaps only those fields; from my observations, peer-review works very well in my field (bchem/chem).
Mrs Draba trained as an Inorganic Chemist. There's a lot of lather/rinse/repeat methodology in Inorganic Chem, but by the same token, peer review seems tick and flick most of the time, since methods don't seem terribly contentious. For contrast, I researched in an area aligned to Artificial Intelligence and that was much messier: competing ontologies, methodologies, even epistemologies -- a constant bunfight to shape the underlying research paradigm. That can get tribal for a while but things still managed to progress and resolve themselves.

However, you can't defy the underlying scientific ethics -- which are to do with accountability, transparency, falsification and honesty. When you do that, you're putting a tinfoil hat firmly on your head, and frolicking off into the realm of pseudoscience.
I would never intend to throw those away; I just don't think that they are inherently tied to only peer-review.
They're not. Formal journal-paper peer-review is just the end of a long process of accountability (formal and informal) intended to make subjective ideas more objective.

Back on the question of fairness to individual scientists -- sadly, like evolution, it's not. There are always more questions than answers, so it's not hard to move most fields forward somewhat. However intelligence and hard work don't bestow grand destiny in science -- that's something of a popular myth. The few really landmark results that are around to be discovered depend on you being in the right field at the right time and (probably) working in the right institution or at least, being in contact with the right people.
Perhaps I still have too much of a young naiive mentality; get everything done NOW :P
You're in the wrong biz then. You need to get your face on Youtube, preferably with a cat. :)

If I were looking at it from a perspective of 'the good of science' as a whole, the peer-review system is very sturdy.
It's not the part I'm most worried about. But if you wanted to talk about career-paths for young scientists, incentives to share data and not just results, the cynical commercial exploitation of scientific publications, and how the IP system of patents is working, I could rave at length. :)

I suppose the most important advice I can offer here, is don't be afraid of looking a fool. Firstly, research supervision should help protect you from that as you work out what's what. Secondly, even famous scientists like Stephen Hawking can be wrong repeatedly (he's lost something like four public scientific bets that I'm aware of.) Thirdly, all conjectures are intuitive, and all intuitions are subjective. So there's no shame in being wrong for good reasons. So if you're going to be wrong (and you will), the most important thing to do is try to be wrong early: months into a project, not decades into a career. It's much cheaper! And detecting wrong early is really what consultation with your peers is for.

They might be scary-smart, neurotically pedantic obsessive-compulsives -- some with a personality bypass -- but like natural selection -- as long as you're determined enough, and can adapt fast enough, they're really your friends. :)
edgar_winters
Posts: 49
Add as Friend
Challenge to a Debate
Send a Message
11/5/2015 10:07:54 PM
Posted: 1 year ago
At 11/3/2015 5:06:11 PM, ithink-ithink wrote:
Is the peer-review system really what we should be using in science? I am a low-level scientist(MSc), so I just wanted to know what some others may have to say about this.

I know the basis behind why it is used; that the people best certified to validate an incoming scientific paper is those scientist in the relevant field.
But what happens when a paper comes about, that would potentially cause a paradigm shift. If the shift is too far from conventional scientific wisdom, isn't there a much higher probability that the paper would be rejected purely on the basis that the scientists in the field (doing the peer review) can't reconcile their life's work/understanding with the paper.

I gather that the old system through which papers were accepted into journals consisted of a multi-disciplinary panel which judged papers on their fluency, logic, quality of experimental evidence etc. Isn't this system better? People's papers would be judged on their own standalone quality, thereby systematically discouraging consensus and scholasticism, as opposed to the peer-review system which seems to systematically encourage it.

Just wanted to hear people's thoughts. : :

There are many groups in the scientific field who disagree with each other just like religious groups do. It's much easier to listen to one who created everything. At least he doesn't pretend to know something and force all the other groups to believe in him.
Akhenaten
Posts: 854
Add as Friend
Challenge to a Debate
Send a Message
11/8/2015 7:29:13 AM
Posted: 1 year ago
The peer review system is incapable of reviewing ground breaking new research and ideas. They can only validate new ideas which elaborate on or collaborate with old ideas.
Thus, the science community is stuck in a never ending vicious circle of deceit and lies to cover-up previous mistakes and to protect redundant and illogical theories from the past.
When somebody invests decades of their life into supporting a particular theory, then, that person will protect that theory with all their energy and will power. It will become a game of life and death to them that an old theory must survive. Their whole feeling of self-worth and ego preservation is at stake.
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
11/8/2015 8:46:22 PM
Posted: 1 year ago
At 11/8/2015 7:29:13 AM, Akhenaten wrote:
The peer review system is incapable of reviewing ground breaking new research and ideas.

This is why science has so dismally failed to innovate in the last four hundred years. :(

If only science could bring itself to accept the periodic table of elements, genetic inheritance, evolution, the Rutherford model of the atom, nuclear physics, electromagnetic waves, the Bohr model of the atom, quantum mechanics, benzene structures, DNA and epigenetics, we might really be having a crack at bioinformatics and genetic engineering right now.

There's more to be said on this, but regrettably I can't stay. I have to change my mother's leeches before I head off to a witch-burning.
Akhenaten
Posts: 854
Add as Friend
Challenge to a Debate
Send a Message
11/8/2015 9:28:16 PM
Posted: 1 year ago
At 11/8/2015 8:46:22 PM, RuvDraba wrote:

This is why science has so dismally failed to innovate in the last four hundred years. :(

If only science could bring itself to accept the periodic table of elements, genetic inheritance, evolution, the Rutherford model of the atom, nuclear physics, electromagnetic waves, the Bohr model of the atom, quantum mechanics, benzene structures, DNA and epigenetics, we might really be having a crack at bioinformatics and genetic engineering right now.

There's more to be said on this, but regrettably I can't stay. I have to change my mother's leeches before I head off to a witch-burning.

Half the so called "breakthroughs" that you have quoted are garbage science. Rutherford model of the atom is just a bad guess at best; quantum mechanics is nonsense and Bohr model of atom worse.

Note - Witch burning was a practised to stop natural healing and facilitated the introduction of the pharmaceutical industry. Science has a lot to answer for, in regards to that matter. lol

See my post 'germ theory is a fraud' for more information!
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
11/8/2015 9:39:32 PM
Posted: 1 year ago
At 11/8/2015 9:28:16 PM, Akhenaten wrote:
At 11/8/2015 8:46:22 PM, RuvDraba wrote:

This is why science has so dismally failed to innovate in the last four hundred years. :(

If only science could bring itself to accept the periodic table of elements, genetic inheritance, evolution, the Rutherford model of the atom, nuclear physics, electromagnetic waves, the Bohr model of the atom, quantum mechanics, benzene structures, DNA and epigenetics, we might really be having a crack at bioinformatics and genetic engineering right now.

There's more to be said on this, but regrettably I can't stay. I have to change my mother's leeches before I head off to a witch-burning.


Half the so called "breakthroughs" that you have quoted are garbage science.

'Garbage' meaning: imprecise models improved through diligent and systematic falsification resulting in a massive reduction of infant morality, eradication of lethal epidemics and crippling diseases , and the extension of human lifespan and activity through old age?
Akhenaten
Posts: 854
Add as Friend
Challenge to a Debate
Send a Message
11/8/2015 10:33:14 PM
Posted: 1 year ago
At 11/8/2015 9:39:32 PM, RuvDraba wrote:

'Garbage' meaning: imprecise models improved through diligent and systematic falsification resulting in a massive reduction of infant morality, eradication of lethal epidemics and crippling diseases , and the extension of human lifespan and activity through old age?

False assumption - That the average life span of humans has improved through modern medicine. Humans were living to 80-90 years of age long before medicine came along. It was city living that reduced the average life span of humans. Humans got used to living to around 30-40 before carking it early. This is because the city diet of stale grain food is unsuitable for humans and the meat was mostly rotten before it could be transported from the country areas. They didn't have refrigeration in the early days, so most food went bad before people ate it. Thus, the short life expectancy and numerous 'disease' epidemics.
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
11/8/2015 11:24:44 PM
Posted: 1 year ago
At 11/8/2015 10:33:14 PM, Akhenaten wrote:
At 11/8/2015 9:39:32 PM, RuvDraba wrote:

'Garbage' meaning: imprecise models improved through diligent and systematic falsification resulting in a massive reduction of infant morality, eradication of lethal epidemics and crippling diseases , and the extension of human lifespan and activity through old age?

False assumption - That the average life span of humans has improved through modern medicine. Humans were living to 80-90 years of age long before medicine came along.

It's true that in ancient times, people could live to 80+ years.

Yet it's also true that the average lifespan has increased. Meaning: more people are living to 80+ than used to.

They're also a lot more active, thanks to hip replacements, spectacles, cataract surgery, hearing-aids, dentures and the like. I know 70+ year-olds who scuba-dive, bicycle, garden... they're not all just hobbling around on sticks.

Even more significant are vastly fewer deaths before the age of three years thanks to medicine, nutrition and hygeine.

Statistics, eh! O.O
Akhenaten
Posts: 854
Add as Friend
Challenge to a Debate
Send a Message
11/9/2015 5:12:14 AM
Posted: 1 year ago
At 11/8/2015 11:24:44 PM, RuvDraba wrote:

It's true that in ancient times, people could live to 80+ years.

Yet it's also true that the average lifespan has increased. Meaning: more people are living to 80+ than used to.

They're also a lot more active, thanks to hip replacements, spectacles, cataract surgery, hearing-aids, dentures and the like. I know 70+ year-olds who scuba-dive, bicycle, garden... they're not all just hobbling around on sticks.

Even more significant are vastly fewer deaths before the age of three years thanks to medicine, nutrition and hygeine.

Statistics, eh! O.O

As long as you don't give your baby any dairy, sugar or grain food, it should do just fine. Whooping cough is a result of dairy, sugar and grain food. Most of the problems that old people have are from eating grain, sugar and dairy products. Loss of hearing, sight, arthritis and mobility is a result of bad diet causing blood supply blockages.
Envisage
Posts: 3,646
Add as Friend
Challenge to a Debate
Send a Message
11/22/2015 8:15:27 PM
Posted: 1 year ago
At 11/3/2015 5:06:11 PM, ithink-ithink wrote:
Is the peer-review system really what we should be using in science? I am a low-level scientist(MSc), so I just wanted to know what some others may have to say about this.

I know the basis behind why it is used; that the people best certified to validate an incoming scientific paper is those scientist in the relevant field.
But what happens when a paper comes about, that would potentially cause a paradigm shift. If the shift is too far from conventional scientific wisdom, isn't there a much higher probability that the paper would be rejected purely on the basis that the scientists in the field (doing the peer review) can't reconcile their life's work/understanding with the paper.

Paradigm shifts have happened countless times in the past in science, most of them you are not aware of. Virtually every significant paper (one that garners many times more citations than otherwise normal) is itself a paradigm shift in its specific field. Peer review has very little to do with the actual claims of the paper, but whether the paper has sufficient integrity in its scholarship and its data to back up the claims made.

For example, I have had several papers myself rejected because of insufficient evidence, and others because it was not relevant enough to the journal.

As for reconciliation, I have done literature reviews with citing many papers that make diametrically opposite claims for the same phenomenon, both of those claims are evidence-based. It's really not uncommon for things like to crop up - that's where science gets done.

I gather that the old system through which papers were accepted into journals consisted of a multi-disciplinary panel which judged papers on their fluency, logic, quality of experimental evidence etc. Isn't this system better? People's papers would be judged on their own standalone quality, thereby systematically discouraging consensus and scholasticism, as opposed to the peer-review system which seems to systematically encourage it.

One probably is that the peer-review system is not universal amongst journals. Difference journals have different peer review systems, of varying effectiveness. There are many what are called 'junk journals', where it is relatively very easy to get poor quality science accepted, and others (such as 'Science' and 'Nature') where it is exceptionally hard to even get high quality science accepted. We just don't have a completely formal system for it.

Just wanted to hear people's thoughts.
PGA
Posts: 4,032
Add as Friend
Challenge to a Debate
Send a Message
11/26/2015 4:21:15 AM
Posted: 1 year ago
At 11/22/2015 8:15:27 PM, Envisage wrote:
At 11/3/2015 5:06:11 PM, ithink-ithink wrote:
Is the peer-review system really what we should be using in science? I am a low-level scientist(MSc), so I just wanted to know what some others may have to say about this.

I know the basis behind why it is used; that the people best certified to validate an incoming scientific paper is those scientist in the relevant field.
But what happens when a paper comes about, that would potentially cause a paradigm shift. If the shift is too far from conventional scientific wisdom, isn't there a much higher probability that the paper would be rejected purely on the basis that the scientists in the field (doing the peer review) can't reconcile their life's work/understanding with the paper.

Paradigm shifts have happened countless times in the past in science, most of them you are not aware of. Virtually every significant paper (one that garners many times more citations than otherwise normal) is itself a paradigm shift in its specific field. Peer review has very little to do with the actual claims of the paper, but whether the paper has sufficient integrity in its scholarship and its data to back up the claims made.

For example, I have had several papers myself rejected because of insufficient evidence, and others because it was not relevant enough to the journal.

As for reconciliation, I have done literature reviews with citing many papers that make diametrically opposite claims for the same phenomenon, both of those claims are evidence-based. It's really not uncommon for things like to crop up - that's where science gets done.

I gather that the old system through which papers were accepted into journals consisted of a multi-disciplinary panel which judged papers on their fluency, logic, quality of experimental evidence etc. Isn't this system better? People's papers would be judged on their own standalone quality, thereby systematically discouraging consensus and scholasticism, as opposed to the peer-review system which seems to systematically encourage it.

One probably is that the peer-review system is not universal amongst journals. Difference journals have different peer review systems, of varying effectiveness. There are many what are called 'junk journals', where it is relatively very easy to get poor quality science accepted, and others (such as 'Science' and 'Nature') where it is exceptionally hard to even get high quality science accepted. We just don't have a completely formal system for it.

Just wanted to hear people's thoughts.

You get younger by the day. Your profile says 14 years old!

We are looking for candidates for some kind of comprehensive debate on the early or late dating of the NT. Are you game or do you know of anyone who is? Ruv is going on vacation and suggested your name. You can consider it here:

http://www.debate.org...

Peter
Envisage
Posts: 3,646
Add as Friend
Challenge to a Debate
Send a Message
11/26/2015 10:14:09 PM
Posted: 1 year ago
At 11/26/2015 4:21:15 AM, PGA wrote:
At 11/22/2015 8:15:27 PM, Envisage wrote:
At 11/3/2015 5:06:11 PM, ithink-ithink wrote:
Is the peer-review system really what we should be using in science? I am a low-level scientist(MSc), so I just wanted to know what some others may have to say about this.

I know the basis behind why it is used; that the people best certified to validate an incoming scientific paper is those scientist in the relevant field.
But what happens when a paper comes about, that would potentially cause a paradigm shift. If the shift is too far from conventional scientific wisdom, isn't there a much higher probability that the paper would be rejected purely on the basis that the scientists in the field (doing the peer review) can't reconcile their life's work/understanding with the paper.

Paradigm shifts have happened countless times in the past in science, most of them you are not aware of. Virtually every significant paper (one that garners many times more citations than otherwise normal) is itself a paradigm shift in its specific field. Peer review has very little to do with the actual claims of the paper, but whether the paper has sufficient integrity in its scholarship and its data to back up the claims made.

For example, I have had several papers myself rejected because of insufficient evidence, and others because it was not relevant enough to the journal.

As for reconciliation, I have done literature reviews with citing many papers that make diametrically opposite claims for the same phenomenon, both of those claims are evidence-based. It's really not uncommon for things like to crop up - that's where science gets done.

I gather that the old system through which papers were accepted into journals consisted of a multi-disciplinary panel which judged papers on their fluency, logic, quality of experimental evidence etc. Isn't this system better? People's papers would be judged on their own standalone quality, thereby systematically discouraging consensus and scholasticism, as opposed to the peer-review system which seems to systematically encourage it.

One probably is that the peer-review system is not universal amongst journals. Difference journals have different peer review systems, of varying effectiveness. There are many what are called 'junk journals', where it is relatively very easy to get poor quality science accepted, and others (such as 'Science' and 'Nature') where it is exceptionally hard to even get high quality science accepted. We just don't have a completely formal system for it.

Just wanted to hear people's thoughts.

You get younger by the day. Your profile says 14 years old!

We are looking for candidates for some kind of comprehensive debate on the early or late dating of the NT. Are you game or do you know of anyone who is? Ruv is going on vacation and suggested your name. You can consider it here:

http://www.debate.org...

Peter

I am not interested. I appreciate the offer however.
PGA
Posts: 4,032
Add as Friend
Challenge to a Debate
Send a Message
11/26/2015 10:46:05 PM
Posted: 1 year ago
At 11/26/2015 10:14:09 PM, Envisage wrote:
At 11/26/2015 4:21:15 AM, PGA wrote:
At 11/22/2015 8:15:27 PM, Envisage wrote:
At 11/3/2015 5:06:11 PM, ithink-ithink wrote:
Is the peer-review system really what we should be using in science? I am a low-level scientist(MSc), so I just wanted to know what some others may have to say about this.

I know the basis behind why it is used; that the people best certified to validate an incoming scientific paper is those scientist in the relevant field.
But what happens when a paper comes about, that would potentially cause a paradigm shift. If the shift is too far from conventional scientific wisdom, isn't there a much higher probability that the paper would be rejected purely on the basis that the scientists in the field (doing the peer review) can't reconcile their life's work/understanding with the paper.

Paradigm shifts have happened countless times in the past in science, most of them you are not aware of. Virtually every significant paper (one that garners many times more citations than otherwise normal) is itself a paradigm shift in its specific field. Peer review has very little to do with the actual claims of the paper, but whether the paper has sufficient integrity in its scholarship and its data to back up the claims made.

For example, I have had several papers myself rejected because of insufficient evidence, and others because it was not relevant enough to the journal.

As for reconciliation, I have done literature reviews with citing many papers that make diametrically opposite claims for the same phenomenon, both of those claims are evidence-based. It's really not uncommon for things like to crop up - that's where science gets done.

I gather that the old system through which papers were accepted into journals consisted of a multi-disciplinary panel which judged papers on their fluency, logic, quality of experimental evidence etc. Isn't this system better? People's papers would be judged on their own standalone quality, thereby systematically discouraging consensus and scholasticism, as opposed to the peer-review system which seems to systematically encourage it.

One probably is that the peer-review system is not universal amongst journals. Difference journals have different peer review systems, of varying effectiveness. There are many what are called 'junk journals', where it is relatively very easy to get poor quality science accepted, and others (such as 'Science' and 'Nature') where it is exceptionally hard to even get high quality science accepted. We just don't have a completely formal system for it.

Just wanted to hear people's thoughts.

You get younger by the day. Your profile says 14 years old!

We are looking for candidates for some kind of comprehensive debate on the early or late dating of the NT. Are you game or do you know of anyone who is? Ruv is going on vacation and suggested your name. You can consider it here:

http://www.debate.org...

Peter

I am not interested. I appreciate the offer however.

Thanks!