Following the mixed method (Creswell, 2008; Johnston and Onwuegbuzie, 2004) in this research, we employed two tools: pre- and post-questionnaires and students' forum discourse analysis.

We established research trustworthiness through (a) investigator triangulation, were three experts in science education research and one in bio-medical engineering conducted content analysis of students’ responses, reaching 90% inter-rater consent, and (b) data triangulation, using the two data sources—the questionnaires and the discourse (Denzin & Lincoln, 2005).

Research Tools and Data Analysis

The questionnaires

The pre- and post-questionnaires were based on adapted scientific articles and aimed at determining differences in students' scientific literacy skills before and after each course. The adapted scientific articles in both the pre- and the post-questionnaires for each of the two courses were shortened versions of the originals, which still retained the canonical structure of a research article. Each adapted article was followed by three open-ended questions. Each one of the four (two pre- and two post-) questionnaires was based on a different scientific article, but the accompanying open-ended questions were similar in all four questionnaires.

These questions focused on three aspects of scientific literacy: (1) Question posing – the students were asked to raise two questions concerning the article they had just read, for which the answer did not appear in the text; (2) Identifying the article structure in terms of matching the type of heading from a given list (introduction, methods, findings, etc.) for each paragraph in the adapted article, and (3) Experiment design – the students were asked to suggest a follow-up experiment based on the experiment described in the article. Each answer in the pre- and post-questionnaires was analyzed using content analysis, descriptive statistics and statistical analysis.

We noted that the number of students in the analyzed sample for the statistical data analysis was lower than the number of students who participated in every course. According to the Technion regulations, students' mobility between courses is permitted during the first three weeks of the semester. This explains why students who joined the course later did not answer the pre-questionnaire, while students who left the course did not answer the post-questionnaire.    

We will now introduce two tables and an example, which present the rubrics of the three skills we graded in the questionnaires. In Table 2 we introduce the indications for low and high levels of the question posing scientific literacy skill.

Table 2. Indications for low and high levels of the question posing scientific literacy skill

Example

Level of the example

Description of the categories

Which factors affect the cell remodeling?

Low

 

  • Interdisciplinary-only biology
  • Question content-evolved from the phenomenon in the text
  • Thinking level-Knowledge, Student has to find and know the information to the question.
  • Understanding and organization level-cell and molecular level.

Would using stem cells other than MSCS* affect the experiment results?

High

 

  • Interdisciplinary-biotechnology
  •  Question content-Leads to a well written research question. The answer is not provided in the text.
  • Thinking level-Analysis, In order to answer the question students have to analyze.
  • Understanding and organization level-cell and molecular level and the macro level of the experiment.

MSCs=Mesenchymal stem cells

The next skill we investigated while analyzing the questionnaires was identifying the canonical research article structure as an indication for students' scientific literacy skill. The students were asked to match the correct title to the suitable paragraph in the article, and this skill was graded according to the percent of suitability between the paragraph and the chosen title.

Below, we present an example for the correct matching of the title discussion to the suitable paragraph in the article. The correct paragraph matched to this title is:

"These findings strongly support the proposition that the primary function of dystrophin is to provide mechanical reinforcement to the sarcolemma and thereby protect it from the membrane stresses developed during muscle contraction. Furthermore, the methodology used in this study should prove useful in assessing the efficacy of dystrophin gene therapy in the mdx mouse."

An example of a student who matched the title discussion to the incorrect paragraph is presented below:

"If the primary function of dystrophin is to provide structural integrity to the muscle membrane, one would predict that the major determinant of membrane damage would be the level of membrane stress associated with contraction rather than the number of muscle activations. If, on the other hand, the fundamental problem in dystrophin-deficient muscle is an inability to handle Ca+2, then the high Ca+2 load associated with repeated muscle activation should result in the greatest level of damage."

In the last rubrics, we will introduce the analysis of students' scientific literacy as expressed in the skill of experiment design in the questionnaires, as can be seen in Table 3.

Table 3. Indications for low and high levels of the experiment design scientific literacy skill

Example

Level of the example

Description of the categories

I would like to see if doing an experiment similar to this but with one hydrogel would give better results of cell remodeling.

Low

 

  • Relevancy of the design - repetition of what is described in the article
  • Not innovative

 

To check the membrane permeability to Ca+2, check the amount of Ca+2 in the muscle cell after multiple activations.

High

 

  • Relevancy of the design- combines new aspects and extended beyond the experiment discussed in the article.
  • Innovative

 

Students' discourse analysis in the asynchronous forum

In order to trace students' progress in terms of scientific literacy in their discourse in the course website, as well as to validate the findings we have identified analyzing the questionnaires, during the course From Cell to Tissue, the students were obligated to read a research paper every week, and to participate in asynchronous forum discussions on the course internet website. The students were divided into four independent forum groups, each of them limited to about 25 students.

An Internet forum is an online discussion site where people can hold conversations in the form of posted messages. They differ from chat rooms in that messages are at least temporarily archived (Garrison, 2006).

Each week another couple of students in the course From Cell to Tissue had to write down two or three questions for opening the forum discussion, as well as to track and direct the ongoing discussions in the forums. Those students served as peer-tutors for mentoring the ongoing forum discourse. The other students in the course had to respond to their peer mentors' questions and to comment on their peers' responses, accordingly serving as mentees. 

Forums have a specific set of jargon associated with them. A single conversation is called a thread (Garrison, 200). Analyzing the peer-tutor students' raised questions, as well as the accompanied answers by the participant students, namely the forums' threads, was carried out according to the following rubrics. To clarify the categories and their levels, Table 4 presents the students’ questions and answers along with a detailed analysis of the thinking and organization levels [in brackets].

Table 4. Rubrics for analyzing students' scientific literacy, as established in the asynchronous forum questions and answers:

Example of questions and accompanied  answers

Level of the  example

Description of the categories

Q: why woman have higher risk for Osteoporosis?

Low

 

  • Interdisciplinary-medicine
  • Content- The answer to   the  question is provided in the text
  • Thinking level-Seeking information and describing.
  • Understanding and organization level- organism.

 

 

A: According to the paper, in woman about the age of 40 there is about 5% less bone mass in comparison to men. In addition, in menopause there is a systemic decrease in the levels of the estrogen hormone [molecular level], which is responsible for building and protecting the bone mass. Therefore, an advanced process of loosing bone mass starts in women around menopause

Low

  • Interdisciplinary-biology
  • Content- The answer to the question is provided in the text.
  •  Thinking level- Seeking information and describing. organization level - organism level and molecular level

Q: If a growth in the calcium levels causes such damage to mdx cells  we can ask ourselves what is the mechanism which enable healthy cells overcome such change?

High

 

  •  Interdisciplinary- biology and chemistry
  • Content- the answer is not explicit in the text
  • Thinking level-Analysis Organization level- cell molecular and mechanism level.

A: Healthy cells know how to remove from extra amount of calcium using charge channels in the membrane Accordingly when in mdx cells there is damage in the sarcolemma (tears) , the channels will not work efficiently in balancing the calcium level in the cell (this is a toxic situation  for the cells). Due to high levels of calcium in the cell the cell regulation on processes such as muscle construction will be damaged

High

 

  • Interdisciplinary- biology and chemistry
  • Content- the answer is not explicit in the text
  • Thinking level-Analysis and conclusion.
  • Understanding and organization level- cell molecular and mechanism level.