原创

journalnode Can't scan a pre-transactional edit log异常处理

版权声明:本文为博主原创文章,遵循 CC 4.0 BY 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://lidong.blog.csdn.net/article/details/79628425

一个测试环境hadoop集群由于磁盘满导致宕机,启动后发现journalnode报如下异常:

2018-03-19 20:48:04,817 WARN  namenode.FSImage (EditLogFileInputStream.java:scanEditLog(359)) - Caught exception after scanning through 0 ops from /data1_4T/journal/mycluster/current/edits_inprogress_0000000000024973700 while determining its valid length. Position was 1011712

java.io.IOException: Can't scan a pre-transactional edit log.
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogOp$LegacyReader.scanOp(FSEditLogOp.java:4974)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.scanNextOp(EditLogFileInputStream.java:245)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.scanEditLog(EditLogFileInputStream.java:355)
        at org.apache.hadoop.hdfs.server.namenode.FileJournalManager$EditLogFile.scanLog(FileJournalManager.java:551)
        at org.apache.hadoop.hdfs.qjournal.server.Journal.scanStorageForLatestEdits(Journal.java:192)
        at org.apache.hadoop.hdfs.qjournal.server.Journal.<init>(Journal.java:152)
        at org.apache.hadoop.hdfs.qjournal.server.JournalNode.getOrCreateJournal(JournalNode.java:90)
        at org.apache.hadoop.hdfs.qjournal.server.JournalNode.getOrCreateJournal(JournalNode.java:99)
        at org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.getEditLogManifest(JournalNodeRpcServer.java:189)
        at org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.getEditLogManifest(QJournalProtocolServerSideTranslatorPB.java:224)
        at org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:25431)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject

【解决过程】

    namenode是无法启动的,发现JournalNode维护的edits文件损坏,3台JournalNode只有1台上的JournalNode的运行日志是正常的,初步判断这个JournalNode可用,我们把这个JournalNode的数据文件copy到其他两个上去

1、停集群服务

2、将JournalNode异常的2个节点上的数据文件备份移到别的目录,然后删除JournalNode数据文件,copy正常JournalNode的数据文件到这2台节点上来

3、修改copy过来的这些文件的权限

4、启动服务,恢复正常

展开阅读全文

You can't operate on a closed Statement!!!

02-25

private static final String QUERY_PROCESS ="select * from t_wf_process where uuid =?"; rn @Overridern public ProcessEntity getProcessEntity(String processUUID)throws SQLException rn ProcessEntity processEntity = null;rn try rn connection = this.commonJDBC.getConnection();rn ps = connection.prepareStatement(QUERY_PROCESS);rn ps.setString(1, processUUID);rn rs = ps.executeQuery();rn if (rs.next()) rn processEntity = WFEngineConstant.fillProcessEntity(rs);rn else rn processEntity = null;rn rn catch (SQLException e) rn logger.info("getProcessEntity出现异常", e);rn finally rn this.commonJDBC.cleanUp(connection, ps, rs); // 释放资源rn rn // 返回流程实例对象rn return processEntity;rn rn//获取连接*************rn /**rn * 获取Connectionrn */rn public Connection getConnection()rn Connection connection = this.template.getConnection();rn try rn if (connection.isClosed()) rn connection = SqlSessionUtils.getSqlSession(rn this.template.getSqlSessionFactory(),rn this.template.getExecutorType(),rn this.template.getPersistenceExceptionTranslator())rn .getConnection();rn rn catch (Exception e) rn e.printStackTrace();rn rn return connection; rn rnrn//关闭********rnrn/**rn * 关闭Connection,PreparedStatement,ResultSetrn */rn public void cleanUp(Connection c,PreparedStatement ps,ResultSet rs)rn this.closeResultSet(rs);rn this.closePreparedStatement(ps);rn this.closeConnection(c);rn rn rn /**rn * 关闭Connectionrn */rn public void closeConnection(Connection c)rn try rn if (c != null) rn c.close();rn rn catch (Exception e) rn e.printStackTrace();rn rn rn rn /**rn * 关闭PreparedStatementrn */rn public void closePreparedStatement(PreparedStatement ps)rn try rn if (ps != null) rn ps.close();rn rn catch (Exception e) rn e.printStackTrace();rn rn rn rn /**rn * 关闭ResultSetrn */rn public void closeResultSet(ResultSet rs)rn try rn if (rs != null) rn rs.close();rn rn catch (Exception e) rn e.printStackTrace();rn rn rn使用的是c3p0 注入template 模板得到连接的rn有时候抛出如下的错。。。但是有时候又没有这样的错。。。是什么原因。。rnrn抛出getProcessEntity出现异常!rnYou can't operate on a closed Statement!!!rn 论坛

Can't fetch history log; missing job ID Can't fetch task logs; missing applicati

06-04

[code=python]rnpython /home/hduser/Workspace/Ch5_Py_mrjob_PegasosSVM/mrSVM.py -r hadoop < Workspace/Ch5_Py_mrjob_PegasosSVM/kickStart.txtrn[/code]rnrn之前顺利执行过一次,也尝试过格式话namenode;重新删除建立data文件夹,问题仍然存在:rn报错:rnNo handlers could be found for logger “mrjob.launch”rnNo configs found; falling back on auto-configurationrnNo configs specified for hadoop runnerrnLooking for hadoop binary in /usr/local/hadoop/bin…rnFound hadoop binary: /usr/local/hadoop/bin/hadooprnUsing Hadoop version 2.7.5rnLooking for Hadoop streaming jar in /usr/local/hadoop…rnFound Hadoop streaming jar: /usr/local/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.5.jarrnCreating temp directory /tmp/mrSVM.hduser.20180604.015226.736070rnreading from STDINrnCopying local files to hdfs:///user/hduser/tmp/mrjob/mrSVM.hduser.20180604.015226.736070/files/…rnRunning step 1 of 2…rnsession.id is deprecated. Instead, use dfs.metrics.session-idrnInitializing JVM Metrics with processName=JobTracker, sessionId=rnCannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initializedrnCleaning up the staging area file:/usr/local/hadoop/hadoop_data/mapred/staging/hduser854512016/.staging/job_local854512016_0001rnError launching job , bad input path : File does not exist: /usr/local/hadoop/hadoop_data/mapred/staging/hduser854512016/.staging/job_local854512016_0001/files/mrSVM.py#mrSVM.pyrnStreaming Command Failed!rnAttempting to fetch counters from logs…rnCan’t fetch history log; missing job IDrnNo counters foundrnScanning logs for probable cause of failure…rnCan’t fetch history log; missing job IDrnCan’t fetch task logs; missing application IDrnStep 1 of 2 failed: Command ‘[‘/usr/local/hadoop/bin/hadoop’, ‘jar’, ‘/usr/local/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.5.jar’, ‘-files’, ‘hdfs:///user/hduser/tmp/mrjob/mrSVM.hduser.20180604.015226.736070/files/mrSVM.py#mrSVM.py,hdfs:///user/hduser/tmp/mrjob/mrSVM.hduser.20180604.015226.736070/files/mrjob.zip#mrjob.zip,hdfs:///user/hduser/tmp/mrjob/mrSVM.hduser.20180604.015226.736070/files/setup-wrapper.sh#setup-wrapper.sh’, ‘-input’, ‘hdfs:///user/hduser/tmp/mrjob/mrSVM.hduser.20180604.015226.736070/files/STDIN’, ‘-output’, ‘hdfs:///user/hduser/tmp/mrjob/mrSVM.hduser.20180604.015226.736070/step-output/0000’, ‘-mapper’, ‘sh -ex setup-wrapper.sh python mrSVM.py –step-num=0 –mapper’, ‘-reducer’, ‘sh -ex setup-wrapper.sh python mrSVM.py –step-num=0 –reducer’]’ returned non-zero exit status 512rn[color=#FF00FF][b]各位大神,求指教~[/b][/color] 论坛

why I can't adding a window to a container

07-11

源程序如下:rnrnimport java.awt.*;rnimport java.awt.event.*;rnimport javax.swing.*;rnrnpublic class ComCountry extends JFrame implements ItemListenerrnrn private String[] flagTitle = "Canada", "China", "Denmark",rn "France", "Germany", "India", "Norway", "United Kingdom",rn "United States of America";rn private ImageIcon flagImage[]= new ImageIcon[9];rn private String flagDescription[]=new String[9];rn rn private JComboBox countryNameComboBox=new JComboBox();rn private DescriptionPanel descriptionPanel=new DescriptionPanel();rn rn public ComCountry()rn rn flagImage[0] = new ImageIcon("images/ca.gif");rn flagImage[1] = new ImageIcon("images/china.gif");rn flagImage[2] = new ImageIcon("images/denmark.gif");rn flagImage[3] = new ImageIcon("images/fr.gif");rn flagImage[4] = new ImageIcon("images/germany.gif");rn flagImage[5] = new ImageIcon("images/india.gif");rn flagImage[6] = new ImageIcon("images/norway.gif");rn flagImage[7] = new ImageIcon("images/uk.gif");rn flagImage[8] = new ImageIcon("images/us.gif");rnrn // Set text descriptionrn flagDescription[0] = "The Maple Leaf flag \n\n" +rn "The Canadian National Flag was adopted by the Canadian " +rn "Parliament on October 22, 1964 and was proclaimed into law " +rn "by Her Majesty Queen Elizabeth II (the Queen of Canada) on " +rn "February 15, 1965. The Canadian Flag (colloquially known " +rn "as The Maple Leaf Flag) is a red flag of the proportions " +rn "two by length and one by width, containing in its centre a " +rn "white square, with a single red stylized eleven-point " +rn "mapleleaf centred in the white square.";rn flagDescription[1] = "Description for China ... ";rn flagDescription[2] = "Description for Denmark ... ";rn flagDescription[3] = "Description for France ... ";rn flagDescription[4] = "Description for Germany ... ";rn flagDescription[5] = "Description for India ... ";rn flagDescription[6] = "Description for Norway ... ";rn flagDescription[7] = "Description for UK ... ";rn flagDescription[8] = "Description for US ... ";rn rn JLabel TitleLabel=new JLabel("QZM's first program");rn JPanel p1=new JPanel();rn p1.setLayout(new BorderLayout());rn p1.add(TitleLabel,BorderLayout.CENTER);rn p1.add(countryNameComboBox,BorderLayout.WEST);rn rn getContentPane().setLayout(new BorderLayout());rn getContentPane().add(p1,BorderLayout.NORTH);rn getContentPane().add(descriptionPanel,BorderLayout.CENTER);rn rn countryNameComboBox.addItemListener(this);rn rn rn public static void main(String [] args)rn rn // DescriptionPanel frame1=new DescriptionPanel();rn ComCountry frame1=new ComCountry();rn frame1.setVisible(true);rn frame1.pack();rn frame1.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);rn rn rn public void itemStateChanged(ItemEvent e)rn rn DisplayDescriptionPanel(countryNameComboBox.getSelectedIndex());rn rn rn public void DisplayDescriptionPanel(int index)rn rn descriptionPanel.setImage(flagImage[index]);rn descriptionPanel.setTitle(flagTitle[index]);rn descriptionPanel.setText(flagDescription[index]);rn rnrnrnclass DescriptionPanel extends JFramernrn private JTextArea textArea1=new JTextArea();rn private JLabel countryImageLabel=new JLabel();rn private JLabel countryTitleLabel=new JLabel(); rn public DescriptionPanel()rn rn JPanel p1=new JPanel();rn rn p1.setLayout(new BorderLayout());rn p1.add(countryImageLabel,BorderLayout.CENTER);rn p1.add(countryImageLabel,BorderLayout.SOUTH);rn rn JScrollPane jScrollPane=new JScrollPane();rn jScrollPane.add(textArea1);rn textArea1.setLineWrap(true);rn textArea1.setWrapStyleWord(true);rn rn setLayout(new BorderLayout());rn add(jScrollPane,BorderLayout.EAST);rn add(p1,BorderLayout.CENTER);rn rn rn public void setImage(ImageIcon icon)rn rn countryImageLabel.setIcon(icon);rn Dimension dimension=new Dimension(icon.getIconWidth(),icon.getIconHeight());rn countryImageLabel.setPreferredSize(dimension); rn rn rn public void setTitle(String title)rn rn countryTitleLabel.setText(title);rn rn rn public void setText(String text)rn rn textArea1.setText(text);rn rnrnrn编译 OKrn运行出现如下错误。rnException in thread "main" java.lang.IllegalArgumentException: adding a window to a containerrn at java.awt.Container.addImpl(Container.java:1022)rn at java.awt.Container.add(Container.java:903)rn at ComCountry.(ComCountry.java:55)rn at ComCountry.main(ComCountry.java:62)rn本人系Java初学者,望高手指点。不甚感激。 论坛

没有更多推荐了,返回首页