Total Posts:17|Showing Posts:1-17
Jump to topic:

AMA: What constitutes scientific validity?

RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
2/22/2015 6:23:43 PM
Posted: 1 year ago
Hi. I'm a scientist by training and initial vocation, though not a scientist from one of the more 'contentious' sciences of biology, genetics, astrophysics or climatology. :)

My PhD was in automated reasoning -- think of Artificial Intelligence trying to prove mathematical truths. But I've taught science, trained scientists and engineers, and these days I consult to government on how to collect the information they need to make robust policy decisions. I suppose my field has given me a lot of exposure to what constitutes hard evidence and validity.

I'm fairly new to DDO, but I've seen a lot of argument here and in the Religion forum about what truth means and how we recognise it. So while I can't answer every scientific question, I'm happy to talk about what constitutes evidence and conclusive evidence in science, and what happens when science finds something unexpected.

This thread isn't just for the religious -- it's for anyone interested, but please feel free to ask me anything related. I'll answer if I can.
UndeniableReality
Posts: 1,897
Add as Friend
Challenge to a Debate
Send a Message
2/22/2015 10:19:08 PM
Posted: 1 year ago
At 2/22/2015 6:23:43 PM, RuvDraba wrote:
Hi. I'm a scientist by training and initial vocation, though not a scientist from one of the more 'contentious' sciences of biology, genetics, astrophysics or climatology. :)

My PhD was in automated reasoning -- think of Artificial Intelligence trying to prove mathematical truths. But I've taught science, trained scientists and engineers, and these days I consult to government on how to collect the information they need to make robust policy decisions. I suppose my field has given me a lot of exposure to what constitutes hard evidence and validity.

I'm fairly new to DDO, but I've seen a lot of argument here and in the Religion forum about what truth means and how we recognise it. So while I can't answer every scientific question, I'm happy to talk about what constitutes evidence and conclusive evidence in science, and what happens when science finds something unexpected.

This thread isn't just for the religious -- it's for anyone interested, but please feel free to ask me anything related. I'll answer if I can.

What would you say is required of someone in order for them to be considered a scientist?
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
2/22/2015 10:33:20 PM
Posted: 1 year ago
At 2/22/2015 10:19:08 PM, UndeniableReality wrote:

What would you say is required of someone in order for them to be considered a scientist?

It isn't enough to have read scientific literature, UR, or have a chemistry set at home.

Typically, the minimum criteria you'd need to meet are those you'd typically demonstrate through a PhD, comprising:

* a knowledge and critical understanding of the history and literature of a field;
* the ability to identify key unresolved issues, and develop scientifically sound approaches to investigating them;
* having produced an original, significant and scientifically rigorous contribution to the discipline; and
* the demonstrated critical facility to evaluate such a contribution to explain its position, relevance, residual issues and to map out any outstanding work.

That gets you in at the entry level -- it makes you a journeyman, if you will, able to conduct your own research with little or no supervision, but not necessarily an expert. I should also mention that the formal qualification of a PhD isn't strictly necessary. Many demonstrate their capacity through other means (e.g. by contributing to a research team.) And some researchers have no degrees but a real aptitude for research, and some Masters-by-research graduates are very good scientists too.

But those abilities are reviewed by the discipline itself -- much as accomplished doctors have to examine medical interns.

But once you're a scientist, all that does is let you participate in the community. Essentially, everything you claim thereafter in scientific publication will be examined and critiqued by your peers for the rest of your professional life. There are (or should be) no unexamined pronouncements from the mount, and being right yesterday doesn't make you right today.

So the worship of individual scientists which sometimes occurs (Einstein being a conspicuous example) is more a product of Hollywood romance and popular journalism. Since is full of curious personalities and has seen some remarkable insights, but it's actually a profession of congenial vivisectors, politely pulling each other's ideas apart, until what's left is very, very robust. :D
UndeniableReality
Posts: 1,897
Add as Friend
Challenge to a Debate
Send a Message
2/22/2015 10:43:09 PM
Posted: 1 year ago
At 2/22/2015 10:33:20 PM, RuvDraba wrote:
At 2/22/2015 10:19:08 PM, UndeniableReality wrote:

What would you say is required of someone in order for them to be considered a scientist?

It isn't enough to have read scientific literature, UR, or have a chemistry set at home.

Of course. I hope you didn't think I had suggested this.


Typically, the minimum criteria you'd need to meet are those you'd typically demonstrate through a PhD, comprising:

* a knowledge and critical understanding of the history and literature of a field;
* the ability to identify key unresolved issues, and develop scientifically sound approaches to investigating them;
* having produced an original, significant and scientifically rigorous contribution to the discipline; and
* the demonstrated critical facility to evaluate such a contribution to explain its position, relevance, residual issues and to map out any outstanding work.

That gets you in at the entry level -- it makes you a journeyman, if you will, able to conduct your own research with little or no supervision, but not necessarily an expert. I should also mention that the formal qualification of a PhD isn't strictly necessary. Many demonstrate their capacity through other means (e.g. by contributing to a research team.) And some researchers have no degrees but a real aptitude for research, and some Masters-by-research graduates are very good scientists too.

But those abilities are reviewed by the discipline itself -- much as accomplished doctors have to examine medical interns.

But once you're a scientist, all that does is let you participate in the community. Essentially, everything you claim thereafter in scientific publication will be examined and critiqued by your peers for the rest of your professional life. There are (or should be) no unexamined pronouncements from the mount, and being right yesterday doesn't make you right today.

So the worship of individual scientists which sometimes occurs (Einstein being a conspicuous example) is more a product of Hollywood romance and popular journalism. Since is full of curious personalities and has seen some remarkable insights, but it's actually a profession of congenial vivisectors, politely pulling each other's ideas apart, until what's left is very, very robust. :D

This was a good answer. I'm also a scientist (still actively doing research), and I just wanted to hear your perspective. I should add that I'm a fairly new scientist who's only published a few papers so far, and I haven't finished my PhD yet. But I think once your'e actively publishing and part of multiple research teams, leading projects, and teaching graduate courses (in AI no less, so we have something else in common), you can call yourself a scientist. I guess getting paid a salary for doing research helps too, albeit in a trivial sense.

Thanks for the response!
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
2/22/2015 11:02:10 PM
Posted: 1 year ago
At 2/22/2015 10:43:09 PM, UndeniableReality wrote:
I hope you didn't think I had suggested this.

No, you didn't, but I think we have a broad readership here, and just as there are armchair quarterbacks and weekend car-mechanics I think there are plenty of amateur would-be scientists. :D

Actually, the role of amateurs in science is really rich these days. A scientist friend has recently retired from the field of biology and taken up astronomy as a hobby. Scientists are now crowd-sourcing astronomical observations, and I hear that biologists are doing the same with species observations. I love the democratisation of observation and experiment, and think it's great that anyone can now contribute to science -- even if not everyone can run the full end-to-end investigations.

I think once your'e actively publishing and part of multiple research teams, leading projects, and teaching graduate courses (in AI no less, so we have something else in common), you can call yourself a scientist. I guess getting paid a salary for doing research helps too, albeit in a trivial sense.

Yes, broadly speaking I agree. Among my doctoral students I found the hardest skill to pick up was the ability to hive off an original problem of the right size and maturity. This is the sort of problem one doesn't have when working in a research team, but it's a critical professional skill to develop. If you already have the experimental disciplines and the field knowledge, then once you have that and can show you can self-direct a successful project, you're set. :)

Under another hat I also write fiction, and the question of 'when should you call yourself a writer' comes up there. A simple social criterion is: when professional writers are reading your fiction by choice, you're a writer. Likewise, I think that paid or not, full-time or not, when professional scientists are reading your papers by choice, you're a scientist. :)
UndeniableReality
Posts: 1,897
Add as Friend
Challenge to a Debate
Send a Message
2/22/2015 11:13:11 PM
Posted: 1 year ago
At 2/22/2015 11:02:10 PM, RuvDraba wrote:
At 2/22/2015 10:43:09 PM, UndeniableReality wrote:
I hope you didn't think I had suggested this.

No, you didn't, but I think we have a broad readership here, and just as there are armchair quarterbacks and weekend car-mechanics I think there are plenty of amateur would-be scientists. :D

Good point.


Actually, the role of amateurs in science is really rich these days. A scientist friend has recently retired from the field of biology and taken up astronomy as a hobby. Scientists are now crowd-sourcing astronomical observations, and I hear that biologists are doing the same with species observations. I love the democratisation of observation and experiment, and think it's great that anyone can now contribute to science -- even if not everyone can run the full end-to-end investigations.

I really like that. I also work with a company and we have the world's largest repository of EEG data because we've provided tools for users/consumers to build and share their own tasks with a commercial EEG headset and upload their data to us (after it has been stripped of identifying information and with their explicit permission) to mine and do research with.

I think once your'e actively publishing and part of multiple research teams, leading projects, and teaching graduate courses (in AI no less, so we have something else in common), you can call yourself a scientist. I guess getting paid a salary for doing research helps too, albeit in a trivial sense.

Yes, broadly speaking I agree. Among my doctoral students I found the hardest skill to pick up was the ability to hive off an original problem of the right size and maturity. This is the sort of problem one doesn't have when working in a research team, but it's a critical professional skill to develop. If you already have the experimental disciplines and the field knowledge, then once you have that and can show you can self-direct a successful project, you're set. :)

Gaining these skills and kinds of experiences quickly is the greatest advantage I've seen that comes with being the first person at your institution to work in a particular field (brain-computer interfacing). You have no choice but to devise and complete original problems from start to finish.


Under another hat I also write fiction, and the question of 'when should you call yourself a writer' comes up there. A simple social criterion is: when professional writers are reading your fiction by choice, you're a writer. Likewise, I think that paid or not, full-time or not, when professional scientists are reading your papers by choice, you're a scientist. :)

This makes a lot of sense. I started thinking about whether I would be considered a scientist when I googled my name and found way more references to myself and my work than I expected. It's a nice feeling to complete a project that took 1-2 years of work and see that people are reading it, discussing it, tweeting the results, citing it, etc.

Btw, glad to have you on DDO.
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
2/22/2015 11:24:42 PM
Posted: 1 year ago
At 2/22/2015 11:13:11 PM, UndeniableReality wrote:
I also work with a company and we have the world's largest repository of EEG data because we've provided tools for users/consumers to build and share their own tasks with a commercial EEG headset and upload their data to us (after it has been stripped of identifying information and with their explicit permission) to mine and do research with.

After starting in formal systems/computational logic and flirting with virtual reality, I moved into information analytics and data mining. I had the pleasure of helping research teams use analytics to investigate questions as varied as tax fraud and black hole gravitational lensing. :)

One of the reasons I love working with government is that they have so much data about people, and so little idea what they might do with it.

There are huge privacy issues associated with personal data. I don't think public policy has begun to scratch the surface on that. But there are also huge social benefits one can offer with intelligent analysis of data collected around some fairly ordinary events.

Meanwhile, Mrs Draba and I are keen cyclists and we're now collecting a truckload of sporting telemetrics -- cardio, pedalling cadence, hill gradients, speed... She's a former chemist, and so she loves the readouts, and analyses her performance rigorously. But in my idle moments I think about how I'd like to process the analytics there, and what additional data I'd like to use. Power output, for example, would be great to have, but the price of a good power meter exceeds the cost of many decent bikes. :)

Btw, glad to have you on DDO.

Thank you, UR! It's a funny old place, but some of the discussions are very interesting. :)
Welfare-Worker
Posts: 1,171
Add as Friend
Challenge to a Debate
Send a Message
2/23/2015 6:36:56 PM
Posted: 1 year ago
At 2/22/2015 6:23:43 PM, RuvDraba wrote:
Hi. I'm a scientist by training and initial vocation, though not a scientist from one of the more 'contentious' sciences of biology, genetics, astrophysics or climatology. :)

My PhD was in automated reasoning -- think of Artificial Intelligence trying to prove mathematical truths. But I've taught science, trained scientists and engineers, and these days I consult to government on how to collect the information they need to make robust policy decisions. I suppose my field has given me a lot of exposure to what constitutes hard evidence and validity.

I'm fairly new to DDO, but I've seen a lot of argument here and in the Religion forum about what truth means and how we recognise it. So while I can't answer every scientific question, I'm happy to talk about what constitutes evidence and conclusive evidence in science, and what happens when science finds something unexpected.

This thread isn't just for the religious -- it's for anyone interested, but please feel free to ask me anything related. I'll answer if I can.

Victorian Stodden and others are drawing attention to a credibility issue with computational Science.
Do you share her concerns, or is she a lone voice in the wilderness?
Do you think steps need to be taken to correct inherent weaknesses in the current system of the Scientific process?
Do you believe these weaknesses as described by Stodden have led to false 'scientific facts'?
Are there more false scientific facts to correct (if there ever were), once the system has been 'cleaned up'?

Dean's Lecture
Wednesday, February 1, 2012, 4:10 pm - 5:30 pm
"Scientific computation is emerging as absolutely central to the scientific method, but the prevalence of very relaxed practices is leading to a credibility crisis affecting many scientific fields. It is impossible to verify most of the results that computational scientists present at conferences and in papers today. Reproducible computational research, in which all details of computations " code and data " are made conveniently available to others, is necessary for a resolution of this crisis. This requires a multifaceted approach including policy solutions, computational tools for data and code dissemination, curation and archiving, and open licensing frameworks such as the Reproducible Research Standard.
"
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
2/23/2015 7:31:41 PM
Posted: 1 year ago
At 2/23/2015 6:36:56 PM, Welfare-Worker wrote:
Victorian Stodden and others are drawing attention to a credibility issue with computational Science.
Do you share her concerns, or is she a lone voice in the wilderness?

That's a great question, WW, and I think it goes further than computer science.

It's true that due to our vastly improved ability to gather and store data electronically, data stores have burgeoned, in size, complexity and diversity. The data become difficult to catalogue and curate; the methods by which they were collected can become murky, and any processing that might have been done (e.g. to clean or collate them) can become uncertain. So this presents challenges with respect to validating the data-sets used to validate the science. That's one problem.

But another is scientific output itself. Even in the field of Artificial Intelligence where I used to research, the proceedings of a single conference could fill the printed telephone directory of a large city. It's very hard for scientists to keep up with the latest literature, and the availability of the internet -- plus some rather nasty practices by some large academic publishers -- mean there's growing pressure and/or temptation for scientists to skip the peer review process, or a least pre-publish preliminary results while they're waiting for review.

Finally, there's the validation of very complex modelling. We know that virtually all software above a certain size contains logical errors -- bugs, if you will. So validating the output of complex computation means we must somehow validate the software, and we know this cannot be done automatically. When we have multiple programs running multiple computations of the same model on the same data, that helps catch any inconsistencies. But in cases where we have big models computed on rare and expensive equipment, or teams unable to share data, that remains a concern.

Those are real informatic challenges facing science today, WW, and they have to be taken seriously, but I don't believe the scientific method is collapsing because of them, and I'm confident that it has not yet collapsed. I'd be happy to explain why in a separate post, if you are interested.
UndeniableReality
Posts: 1,897
Add as Friend
Challenge to a Debate
Send a Message
2/23/2015 7:44:10 PM
Posted: 1 year ago
At 2/23/2015 7:31:41 PM, RuvDraba wrote:
At 2/23/2015 6:36:56 PM, Welfare-Worker wrote:
Victorian Stodden and others are drawing attention to a credibility issue with computational Science.
Do you share her concerns, or is she a lone voice in the wilderness?

That's a great question, WW, and I think it goes further than computer science.

Just fyi, computational science is not computer science. It's more the applied mathematics and numerical simulation applied to problems in any field of science.


It's true that due to our vastly improved ability to gather and store data electronically, data stores have burgeoned, in size, complexity and diversity. The data become difficult to catalogue and curate; the methods by which they were collected can become murky, and any processing that might have been done (e.g. to clean or collate them) can become uncertain. So this presents challenges with respect to validating the data-sets used to validate the science. That's one problem.

But another is scientific output itself. Even in the field of Artificial Intelligence where I used to research, the proceedings of a single conference could fill the printed telephone directory of a large city. It's very hard for scientists to keep up with the latest literature, and the availability of the internet -- plus some rather nasty practices by some large academic publishers -- mean there's growing pressure and/or temptation for scientists to skip the peer review process, or a least pre-publish preliminary results while they're waiting for review.

That's a slight exaggeration, isn't it? Which conference was that large? The last one I went to was NIPS, and it wasn't close to that big. I think the largest is AAAI, with a little over 400 publications at the last conference.


Finally, there's the validation of very complex modelling. We know that virtually all software above a certain size contains logical errors -- bugs, if you will. So validating the output of complex computation means we must somehow validate the software, and we know this cannot be done automatically. When we have multiple programs running multiple computations of the same model on the same data, that helps catch any inconsistencies. But in cases where we have big models computed on rare and expensive equipment, or teams unable to share data, that remains a concern.

I agree, this is a big concern. They generally do their best to get it right (their careers and reputations depend on it to a large extent), but it's very difficult to catch mistakes and to review your work. In many cases, evaluation is intractable. (This is for other readers, as I'm sure you're familiar with all this).


Those are real informatic challenges facing science today, WW, and they have to be taken seriously, but I don't believe the scientific method is collapsing because of them, and I'm confident that it has not yet collapsed. I'd be happy to explain why in a separate post, if you are interested.

You put a lot of time into your posts, and they're quite good. I hope it doesn't burn you out over time.
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
2/23/2015 8:06:06 PM
Posted: 1 year ago
At 2/23/2015 7:44:10 PM, UndeniableReality wrote:

Just fyi, computational science is not computer science. It's more the applied mathematics and numerical simulation applied to problems in any field of science.

Sorry, UR, I was skipping some steps. System validation and verification is software engineering, whose development I put under the heading of computer science, whose insights of course enable computational science. But SE itself sits under informatics -- information management and processing -- which I think is actually where the fundamental problems lie. I say this having done some 20 years of consulting in information management and analytics to government and scientific agencies who've spent decades storing petabytes of data processed by legacy systems they no longer maintain or even own, and have already lost control of their data. :)

That's a slight exaggeration, isn't it? Which conference was that large? The last one I went to was NIPS, and it wasn't close to that big. I think the largest is AAAI, with a little over 400 publications at the last conference.

AI may have cooled off since I was researching in the 90s, back when Marvin Minsky was evangelising, but as I recall we'd get AAAI and IJCAI proceedings printed in nose-font on cigarette-paper, served up in double volumes, each the size of a phone-book. :) I spent all my time just keeping up on formal systems, and hardly had any time to look at (say) neural networks. :) I understand that some of the hotter areas of genetics and pharmacology are looking like that today too.

You put a lot of time into your posts, and they're quite good. I hope it doesn't burn you out over time.

Thanks, UR. I should be processing payroll for my company, but this is more interesting. :)
UndeniableReality
Posts: 1,897
Add as Friend
Challenge to a Debate
Send a Message
2/23/2015 10:34:39 PM
Posted: 1 year ago
At 2/23/2015 8:06:06 PM, RuvDraba wrote:
At 2/23/2015 7:44:10 PM, UndeniableReality wrote:

Just fyi, computational science is not computer science. It's more the applied mathematics and numerical simulation applied to problems in any field of science.

Sorry, UR, I was skipping some steps. System validation and verification is software engineering, whose development I put under the heading of computer science, whose insights of course enable computational science. But SE itself sits under informatics -- information management and processing -- which I think is actually where the fundamental problems lie. I say this having done some 20 years of consulting in information management and analytics to government and scientific agencies who've spent decades storing petabytes of data processed by legacy systems they no longer maintain or even own, and have already lost control of their data. :)


Right, computational science involves computer science, but they're distinct disciplines. I think the question was specifically about computational science, especially given the context of the question.

That's a slight exaggeration, isn't it? Which conference was that large? The last one I went to was NIPS, and it wasn't close to that big. I think the largest is AAAI, with a little over 400 publications at the last conference.

AI may have cooled off since I was researching in the 90s, back when Marvin Minsky was evangelising, but as I recall we'd get AAAI and IJCAI proceedings printed in nose-font on cigarette-paper, served up in double volumes, each the size of a phone-book. :) I spent all my time just keeping up on formal systems, and hardly had any time to look at (say) neural networks. :) I understand that some of the hotter areas of genetics and pharmacology are looking like that today too.

You may be right. In the 90's there were probably significantly fewer conferences, so more publications in each. Also, since the Minsky era, AI has split into automatic logic systems and machine learning.

You put a lot of time into your posts, and they're quite good. I hope it doesn't burn you out over time.

Thanks, UR. I should be processing payroll for my company, but this is more interesting. :)

Beware the time sink that can be this site. Especially if you start engaging with certain subsets of the population here... =)
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
2/23/2015 10:51:41 PM
Posted: 1 year ago
At 2/23/2015 10:34:39 PM, UndeniableReality wrote:

Right, computational science involves computer science, but they're distinct disciplines.

I'd say it the other way: computer science includes computational science, in the sense that (for example) massively parallel computation and discrete mathematics are computer science topics foundational to computational science.

I'd also say that despite its name, computational science isn't science any more than (say) gas chromatography is -- they're both scientific tools; it's what you do with them that makes it science.

I think the question was specifically about computational science, especially given the context of the question.

Yes it was, but I generalised on the way out to informatics. :)

You may be right. In the 90's there were probably significantly fewer conferences, so more publications in each. Also, since the Minsky era, AI has split into automatic logic systems and machine learning.

It was already starting to split when I was working in it. I worked across both, but principally in the former.

Beware the time sink that can be this site. Especially if you start engaging with certain subsets of the population here... =)

I'm allergic to idiots, and tend not to repeat myself unless I have a new insight to add. :)
UndeniableReality
Posts: 1,897
Add as Friend
Challenge to a Debate
Send a Message
2/24/2015 12:35:41 AM
Posted: 1 year ago
At 2/23/2015 10:51:41 PM, RuvDraba wrote:
At 2/23/2015 10:34:39 PM, UndeniableReality wrote:

Right, computational science involves computer science, but they're distinct disciplines.

I'd say it the other way: computer science includes computational science, in the sense that (for example) massively parallel computation and discrete mathematics are computer science topics foundational to computational science.

I didn't mean to imply one was subset to the other. But I do not agree that computer science includes computational science. Several aspects of computational science exist outside the realm of computer science.


I'd also say that despite its name, computational science isn't science any more than (say) gas chromatography is -- they're both scientific tools; it's what you do with them that makes it science.

I think then you're referring to the tools of computational science. Mathematical modeling, statistics, and numerical analysis are tools which are used in computational science to solve scientific problems in whichever field.


I think the question was specifically about computational science, especially given the context of the question.

Yes it was, but I generalised on the way out to informatics. :)

You may be right. In the 90's there were probably significantly fewer conferences, so more publications in each. Also, since the Minsky era, AI has split into automatic logic systems and machine learning.

It was already starting to split when I was working in it. I worked across both, but principally in the former.

That's true, but machine learning didn't pick up all that much until the late 90's and the 2000's. Hinton was the major player there. I don't know if you've kept up with the literature since then.


Beware the time sink that can be this site. Especially if you start engaging with certain subsets of the population here... =)

I'm allergic to idiots, and tend not to repeat myself unless I have a new insight to add. :)

An allergy I wouldn't mind having.
Welfare-Worker
Posts: 1,171
Add as Friend
Challenge to a Debate
Send a Message
2/24/2015 6:19:20 AM
Posted: 1 year ago
At 2/23/2015 7:31:41 PM, RuvDraba wrote:
At 2/23/2015 6:36:56 PM, Welfare-Worker wrote:
Victorian Stodden and others are drawing attention to a credibility issue with computational Science.
Do you share her concerns, or is she a lone voice in the wilderness?

That's a great question, WW, and I think it goes further than computer science.

It's true that due to our vastly improved ability to gather and store data electronically, data stores have burgeoned, in size, complexity and diversity. The data become difficult to catalogue and curate; the methods by which they were collected can become murky, and any processing that might have been done (e.g. to clean or collate them) can become uncertain. So this presents challenges with respect to validating the data-sets used to validate the science. That's one problem.

But another is scientific output itself. Even in the field of Artificial Intelligence where I used to research, the proceedings of a single conference could fill the printed telephone directory of a large city. It's very hard for scientists to keep up with the latest literature, and the availability of the internet -- plus some rather nasty practices by some large academic publishers -- mean there's growing pressure and/or temptation for scientists to skip the peer review process, or a least pre-publish preliminary results while they're waiting for review.

Finally, there's the validation of very complex modelling. We know that virtually all software above a certain size contains logical errors -- bugs, if you will. So validating the output of complex computation means we must somehow validate the software, and we know this cannot be done automatically. When we have multiple programs running multiple computations of the same model on the same data, that helps catch any inconsistencies. But in cases where we have big models computed on rare and expensive equipment, or teams unable to share data, that remains a concern.

Those are real informatic challenges facing science today, WW, and they have to be taken seriously, but I don't believe the scientific method is collapsing because of them, and I'm confident that it has not yet collapsed. I'd be happy to explain why in a separate post, if you are interested.

Because of the issues you bring up, the Scientific Method is not operating as instructed. If you do not follow the instructions, it seems to me you should not expect robust outcomes.
I am referring to your suggestion that the SM requires:
"3. Make a prediction which, if false, would invalidate the model.
4. Test the prediction.
You lather, rinse and repeat on steps 3-4 until you think the model is robust, then you submit it to the entire scientific community for review. They in turn will review your observations, measures, modeling, and experimentation
."

As Ms Stoddard says, this is not being done, so there is a crisis of credibility.
You say it is not collapsing on itself, which could mean it is sinking but can be salvaged. You do say there are problems that "have to be taken seriously".
You offer no suggestions that I see.
As far as the public is concerned, it is business as usual.
Those non-scientists who are huge fans of science believe it to be a faultless system, beyond reproach. What is your advice to them?

The SM today relies very heavily on complex computer modeling, not just uses it, but relies on it.
It uses them to replace hands on experiments.
As you point out these replacements for hands on experiments have 'bugs' that need corrected, but peer review is not logistically possible. For several decades we have been using faulty methods to gets results from the SM.
Is this a system problem or not?
Is the SM as it is in operation today more accurate than the one in use in 1970, which did not allow computer modeling for hands on replacement?
Were the hands on experiments also inherently problematic, and needed corrected? Is this 'new and improved' system a fix to even bigger problems in the previous applications of the SM?
v3nesl
Posts: 4,470
Add as Friend
Challenge to a Debate
Send a Message
2/24/2015 9:34:25 AM
Posted: 1 year ago
At 2/22/2015 6:23:43 PM, RuvDraba wrote:
Hi. I'm a scientist by training and initial vocation, though not a scientist from one of the more 'contentious' sciences of biology, genetics, astrophysics or climatology. :)

My PhD was in automated reasoning -- think of Artificial Intelligence trying to prove mathematical truths. But I've taught science, trained scientists and engineers, and these days I consult to government on how to collect the information they need to make robust policy decisions. I suppose my field has given me a lot of exposure to what constitutes hard evidence and validity.

I'm fairly new to DDO, but I've seen a lot of argument here and in the Religion forum about what truth means and how we recognise it. So while I can't answer every scientific question, I'm happy to talk about what constitutes evidence and conclusive evidence in science, and what happens when science finds something unexpected.

This thread isn't just for the religious -- it's for anyone interested, but please feel free to ask me anything related. I'll answer if I can.

What I find interesting is that the question is asked at all. In earlier ages of man, the beauty of science was that it was truth, pure and simple. You either got your ideas right or you got it wrong. Either gravity held the planets in orbit or Newton got it wrong. It had nothing to do with education. It had nothing to do with process. One either got at the truth or got it wrong.

So I find it a profound thing to try to substitute process for truth. The idea that one is a 'real' scientist if you have the proper training, rather than if you get it right, is a profound change in scientific philosophy. And I can't help but think that 90+% of it is dedicated towards propping up evolution, the creation myth of humanism. Obviously you can't prove evolution is true, so you must instead prove that it is the belief of real scientists, so by sleight of hand it becomes something you should accept. Evolution becomes a moral imperative, rather than a scientific one.

And there's a whole list of authorized scientific beliefs these days. You must believe in "climate change", for instance, or else you are a "climate change denier". A heretic, in other words. The fact that taking temperature readings should be about as simple as science gets just highlights that it is in fact a faith based movement, not a data based one.

I've said in the past that the rise of Darwinism marks the twilight of the golden age of science. We are today in the age of technology, not the age of science. The move is indeed towards automated reasoning, isn't it?
This space for rent.
RuvDraba
Posts: 6,033
Add as Friend
Challenge to a Debate
Send a Message
2/24/2015 11:40:55 AM
Posted: 1 year ago
At 2/24/2015 9:34:25 AM, v3nesl wrote:
In earlier ages of man, the beauty of science was that it was truth, pure and simple.

V3nesl, I'd say that for the early centuries of science, science had the wrong narrative about its purpose without realising it, and this narrative hardened through religious attack, only to collapse in the early 20th century.

In the Middle Ages, Europe had virtually no science, however the Muslim world had mathematics, astronomy, medicine and alchemy -- the beginnings of chemistry. Muslim scholars felt then that science was a holy undertaking: an exploration of the truth of God. When science started to emerge again in the European Renaissance, it was as an exploration of the beauty of creation, and the humanists of the day also believed that the beauty of creation was an articulation of the truth of God. In each case, they felt that truth was knowable, observable, communicable and hence once found, it'd be absolute.

However, that didn't last. From Copernicus' work on astronomy in 1543 to Darwin's publication on evolution in 1859, science had to struggle to be allowed to conduct empirical investigation at all. It's an odd fight, because religious tradition wanted to keep its intellectual authority about the world (i.e. be seen as Right), while scientific inquiry just wanted to find better answers. However when one side claiming absolute truth critiques another side claiming something new, it hardens positions. The intellectual contest became 'who has THE truth', when it ought always to have been 'who has the better understanding.'

I think the narrative of science suffered from that. By the end of the 19th century, we were discussing true knowledge replacing false beliefs, with guys like the British chemist Rutherford saying that science would run out of problems to solve. That's tantamount to saying that the models are all so close to perfect that everything can be explained.

But Rutherford lived in nicely ordered colonial empire without nuclear physics, Einsteinian relativity, an expanding universe, Goedel's Incompleteness Theorem, quantum mechanics, genetics or behavioural psychology. The twentieth century was hugely productive for science, but has had so many conceptual revolutions that I think any self-respecting scientist now has to acknowledge that what science pursues isn't final truth, but an increasingly actionable understanding of what is around us and how it works.

But the proposition that science should produce The Truth or it's worthless has its origins in the attacks of religion. Religion has always claimed The Truth, and feels that to be its principle value, so unless science can produce all The Truth and all at once, it doesn't want to cede anything. However, now the fossil fuel industries are using the same excluded-middle attack: if you can't get 100% consensus on anthropogenic climate change from all scientists, it should be ignored.

Science has never really needed more than better answers, and nowadays, I think most self-respecting scientists would say that what they find isn't truth so much as a more robust, more actionable understanding of what's around us and how it works.

That raises the question of which scientific models are likely to change, and how much they're likely to change, and scientific critics are right to raise it. However, I think there's a conflation between changing models, and changing what the models predict.

A better understanding of climate change might reveal mechanisms nobody knew about, or give more precision to prediction, but it won't alter the effects of carbon emissions in the atmosphere, or our knowledge of who put them there.

A better understanding of genetics and primatology might reveal new insights into the lives and experiences of chimpanzees and bonobo apes, but won't change their genetic kinship with us, and won't remove their extant capacity for language, or sacrifice, compassion and grief.

I hope that may help.